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HUMAN  CAPITAL 

Monitoring  of  Safeguards  and  Addressing  Employee 
Perceptions  Are  Key  to  Implementing  a  Civilian 
Performance  Management  System  in  DOD 


Why  GAO  Did  This  Study 

In  2004,  the  Department  of  Defense 
(DOD)  began  implementing  the 
National  Security  Personnel 
System  (NSPS) — a  human  capital 
system  for  DOD  civilians.  NSPS 
significantly  redesigned  the  way 
DOD  civilians  are  hired, 
compensated,  and  promoted.  Pub. 
L.  No.  110-181  mandated  that  GAO 
conduct  reviews  of  the  NSPS 
performance  management  system 
in  calendar  years  2008,  2009,  and 
2010.  In  this  report,  GAO  assessed 
(1)  the  extent  to  which  DOD  has 
implemented  certain  internal 
safeguards  to  ensure  the  fairness, 
effectiveness,  and  credibility  of 
NSPS,  and  monitored  their 
implementation,  and  (2)  how  DOD 
civilian  personnel  perceive  NSPS, 
and  the  actions  DOD  has  taken  to 
address  those  perceptions.  GAO 
analyzed  relevant  documents  and 
employee  survey  results, 
interviewed  DOD  officials,  and 
conducted  discussion  groups  with 
DOD  employees  at  eight  locations 
outside  of  the  continental  United 
States.  Toward  the  end  of  GAO’s 
review,  both  Houses  of  Congress 
passed  proposed  legislation  that,  if 
enacted,  would  terminate  NSPS 
and  require  any  future  performance 
management  system  for  DOD 
civilians  to  include  certain  internal 
safeguards. 


What  GAO  Recommends 


GAO  is  recommending  that  DOD 
evaluate  its  training,  revise  its 
guidance  for  analyzing  ratings,  and 
monitor  the  implementation  of 
safeguards  for  NSPS  or  any  future 
performance  management  system. 
DOD  generally  concurred  with 
GAO’s  recommendations. 

View  GAO-10-102  or  key  components. 

For  more  information,  contact  Brenda  S. 
Farrell  at  (202)  512-3604  orfarrellb@gao.gov. 


What  GAO  Found 

DOD  continues  to  take  steps  to  implement  internal  safeguards  as  part  of 
NSPS,  but  implementation  of  some  safeguards  could  still  be  improved,  and 
continued  monitoring  of  all  safeguards’  implementation  is  needed.  In  general, 
DOD  has  taken  some  steps  to  meet  the  intent  of  each  of  the  safeguards,  and  it 
has  implemented  some  of  the  recommendations  from  GAO’s  September  2008 
report.  However,  opportunities  exist  for  DOD  to  improve  implementation  of 
some  safeguards.  For  example,  DOD  has  not  yet  evaluated  the  effectiveness  of 
the  training  employees  receive,  although  doing  so  could  help  DOD  measure 
the  impact  of  its  training  and  its  progress  toward  achieving  agency  goals.  In 
addition,  DOD  has  not  specified  in  its  guidance  what  process  the  components 
should  follow  to  investigate  and  eliminate  potential  barriers  to  fair  and 
equitable  ratings.  Consequently,  the  components  may  not  follow  a  consistent 
approach  when  investigating  potential  barriers,  which  could  hinder  their 
efforts  to  eliminate  them.  Further,  GAO  previously  noted  that  continued 
monitoring  of  the  safeguards  was  needed  to  ensure  that  DOD’s  actions  were 
effective.  While  DOD  monitors  some  aspects  of  the  system’s  implementation, 
it  does  not  monitor  how  or  the  extent  to  which  the  safeguards  specifically  are 
implemented  across  the  department.  As  a  result,  decision  makers  lack 
information  that  could  be  used  to  determine  whether  the  department’s  actions 
are  effective  and  whether  the  system  is  being  implemented  in  a  fair,  equitable, 
and  credible  manner. 

DOD  civilian  personnel  have  mixed  perceptions  about  NSPS,  and  while  the 
department  has  taken  some  steps  toward  addressing  employee  concerns,  it 
has  not  yet  developed  and  implemented  an  action  plan  to  address  areas  where 
employees  express  negative  perceptions  of  the  system,  as  GAO  recommended 
in  2008.  DOD’s  survey  data  from  2008  revealed  that  overall,  NSPS  employees 
responded  positively  about  some  aspects  of  performance  management,  such 
as  connecting  pay  to  performance,  and  negatively  about  others,  such  as  the 
performance  appraisal  process.  According  to  the  most  recent  survey  data,  the 
negative  perceptions  of  employees  who  worked  under  NSPS  the  longest 
remain  largely  unchanged  from  what  was  reported  by  GAO  in  2008.  Further, 
as  GAO  reported  in  2008,  employees  and  supervisors  continue  to  express 
negative  perceptions  in  discussion  groups  about  NSPS — for  example,  voicing 
concerns  about  the  negative  impact  of  NSPS  on  employees’  motivation  and 
morale,  and  about  the  excessive  amount  of  time  spent  navigating  the 
performance  management  process.  Such  negative  perceptions  are  not 
surprising  given  that  large-scale  organizational  transformations  often  require 
an  adjustment  period  to  gain  employees’  trust  and  acceptance.  DOD  has  taken 
some  steps  to  address  employees’  perceptions  of  NSPS — for  example,  by 
issuing  a  memorandum  with  suggested  actions  the  components  could  take  to 
address  employee  concerns.  However,  DOD  has  not  yet  developed  and 
implemented  an  action  plan  that  fully  meets  the  intent  of  GAO’s  2008 
recommendation.  Specifically,  DOD  has  not  yet  specified  such  things  as  its 
intended  actions,  who  will  be  responsible,  and  the  time  frames  for  these 
actions.  GAO  continues  to  believe  that  implementing  such  a  plan  has  merit. 
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GAO 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


October  28,  2009 
Congressional  Committees 

In  2004,  the  Department  of  Defense  (DOD)  began  implementing  the 
National  Security  Personnel  System  (NSPS) — a  human  capital  system  for 
DOD  civilian  employees.  NSPS  significantly  redesigned  the  rules, 
regulations,  and  processes  that  govern  the  way  in  which  civilian 
employees  are  hired,  compensated,  and  promoted  within  the  department. 
DOD  began  converting  employees  to  NSPS  in  2006  and,  according  to  the 
department,  by  early  2009  had  converted  more  than  200,000  employees  to 
the  system.  However,  a  series  of  events  transpired  in  2009  leaving  the 
system’s  future  uncertain.  Specifically,  in  February  2009,  the  Chairman  of 
the  House  Committee  on  Armed  Services  and  the  Chairman  of  the 
committee’s  Subcommittee  on  Readiness  urged  DOD  to  halt  conversions 
to  NSPS,  highlighting  concerns  over  a  lack  of  transparency  and 
widespread  discontent  with  the  system.  Subsequently,  in  March  2009,  DOD 
and  the  Office  of  Personnel  Management  (OPM)  announced  that  they 
would  suspend  any  further  conversions  of  organizations  to  NSPS  pending 
the  outcome  of  a  review  of  the  system  led  by  the  Defense  Business  Board.1 
In  July  2009,  the  Defense  Business  Board  presented  the  Secretary  of 
Defense  with  the  report  of  its  review  of  NSPS,  which  included 
recommendations  to  initiate  a  reconstruction  of  the  system  and  to 
continue  the  moratorium  on  conversions  to  NSPS  until  reconstruction  is 
complete.  Most  recently,  in  October  2009,  the  proposed  National  Defense 
Authorization  Act  for  Fiscal  Year  2010,  which  passed  both  Houses  of 
Congress,2  contains  provisions  that  would  terminate  NSPS  and  convert 
DOD  civilian  employees  currently  under  the  system  to  previously  existing 
civilian  personnel  systems  no  later  than  January  1,  2012. 

Before  the  enactment  of  the  legislation  that  authorized  NSPS,  we  raised  a 
number  of  critical  issues  about  the  proposed  regulations  for  NSPS  in  a 


'On  May  14,  2009,  the  Deputy  Secretary  of  Defense  requested  that  the  Defense  Business 
Board  form  a  task  group  to  review  NSPS  to  detennine  (1)  if  the  underlying  design 
principles  and  methodology  for  implementation  are  reflected  in  the  NSPS  program 
objectives;  (2)  whether  the  program  objectives  are  being  met;  and  (3)  whether  NSPS  is 
operating  in  a  fair,  transparent,  and  effective  manner. 

2H.R.  2647  and  H.R.  Conf.  Rep.  No.  111-288,  §1113  (2009). 
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series  of  testimonies  in  2003. 3  Since  then,  we  have  provided  congressional 
decision  makers  with  insight  into  DOD’s  processes  to  design  and 
implement  NSPS  and  the  most  significant  challenges  the  department  faces 
in  implementing  the  system.4  While  GAO  supports  human  capital  reform  in 
the  federal  government,  a  key  challenge  is  ensuring  sustained  and 
committed  leadership  for  such  a  major  transformation  effort.  Further, 
moving  too  quickly  or  prematurely  to  implement  such  systems,  whether  at 
DOD  or  elsewhere,  can  significantly  raise  the  risk  of  problems 
implementing  them.6  We  have  also  reported  that  DOD  and  other  federal 
agencies  must  ensure  that  their  performance  management  systems  contain 
appropriate  accountability  mechanisms  and  internal  safeguards  to  ensure 
that  the  systems  are  fair,  effective,  and  credible.6 

To  help  inform  congressional  deliberations  on  the  implementation  of 
NSPS,  the  National  Defense  Authorization  Act  for  Fiscal  Year  2008 
mandated  that  GAO  conduct  reviews  of  NSPS  in  calendar  years  2008,  2009, 
and  2010  to  determine  the  extent  to  which  DOD  has  effectively 
incorporated  certain  accountability  mechanisms  and  internal  safeguards 


3See  GAO,  Defense  Transformation:  Preliminary  Observations  on  DOD’s  Proposed 
Civilian  Personnel  Reforms,  GAO-03-717T  (Washington,  D.C.:  Apr.  29,  2003);  Defense 
Transformation:  DOD’s  Proposed  Civilian  Personnel  System  and  Governmentwide 
Human  Capital  Reform,  GAO-03-741T  (Washington,  D.C.:  May  1,  2003);  and  Human 
Capital:  Building  on  DOD’s  Reform,  Effort  to  Foster  Govemmentwide  Improvements, 
GAO-03-851T  (Washington,  D.C.:  June  4,  2003). 

4See  GAO,  Human  Capital:  DOD  Needs  to  Improve  Implementation  of  and  Address 
Employee  Concerns  about  Its  National  Security  Personnel  System,  GAO-08-773 
(Washington,  D.C.:  Sept.  10,  2008);  Human  Capital:  DOD  Needs  Better  Internal  Controls 
and  Visibility  over  Costs  for  Implementing  Its  National  Security  Personnel  System, 
GAO-07-851  (Washington,  D.C.:  July  16,  2007);  and  Human  Capital:  Observations  on  Final 
Regulations  for  DOD’s  National  Security  Personnel  System,  GAO-06-227T  (Washington, 
D.C.:  Nov.  17,  2005). 

5GAO,  Questions  for  the  Record  Rela  ted  to  the  Implementation  of  the  Departm  en  t  of 
Defense’s  National  Security  Personnel  System,  GAO-09-669R  (Washington,  D.C.:  May  18, 
2009). 

6See  Related  GAO  Products  at  the  end  of  this  report  for  additional  reports  we  have  issued 
related  to  NSPS  and  performance  management  in  the  federal  government. 
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into  the  system  and  to  assess  employee  attitudes  toward  the  system.7  In 
September  2008,  we  issued  our  first  review  of  NSPS,  and  in  April  2009  we 
testified  on  that  assessment  before  the  House  Committee  on  Armed 
Services’  Subcommittee  on  Readiness.8  We  used  the  following  safeguards 
to  assess  NSPS:9 

•  Involve  employees,  their  representatives,  and  other  stakeholders  in  the 
design  of  the  system,  to  include  employees  directly  involved  in 
validating  any  related  implementation  of  the  system. 

•  Assure  that  the  agency’s  performance  management  system  links 
employee  objectives  to  the  agency’s  strategic  plan,  related  goals,  and 
desired  outcomes. 

•  Implement  a  pay-for-performance  evaluation  system  to  better  link 
individual  pay  to  performance,  and  provide  an  equitable  method  for 
appraising  and  compensating  employees. 


7Pub.  L.  No.  110-181,  §  1106(c)  (2008).  Specifically,  section  1106(c)(1)(B)  directs  GAO  to 
conduct  reviews  in  calendar  years  2008  through  2010  to  evaluate  the  extent  to  which  DOD 
has  effectively  implemented  accountability  mechanisms,  including  those  established  in  5 
U.S.C.  §  9902(b)(7)  and  other  internal  safeguards,  and  to  evaluate,  as  well,  the  extent  that 
DOD  undertakes  internal  assessments  or  employee  surveys  to  assess  employee  satisfaction 
with  NSPS.  The  accountability  mechanisms  specified  in  5  U.S.C.  §  9902(b)(7)  include  those 
that  GAO  previously  identified  as  internal  safeguards  key  to  successful  implementation  of 
performance  management  systems.  For  example  see  GAO,  Post-Hearing  Questions  for  the 
Record  Related  to  the  Department  of  Defense’s  National  Security  Personnel  System 
(NSPS),  GAO-06-582R  (Washington,  D.C.:  Mar.  24,  2006).  GAO  has  emphasized  the  need  for 
internal  safeguards  since  DOD  first  proposed  NSPS.  For  example  see  GAO,  Posthearing 
Questions  Related  to  Strategic  Human  Capital  Management,  GAO-03-779R  (Washington, 
D.C.:  May  22,  2003).  Additionally,  our  review  covered  merit  system  principles  that  pertain 
to  performance  management  systems — specifically  those  stipulated  in  6  U.S.C.  § 

2301(b)(2), (3), (7),  and  (8a). 

8See  GAO-08-773  and  GAO,  Human  Capital:  Improved  Implementation  of  Safeguards  and 
an  Action  Plan  to  Address  Employee  Concerns  Could  Increase  Employee  Acceptance  of 
the  National  Security  Persojmel  System,  GAO-09-464T  (Washington,  D.C.:  Apr.  1,  2009). 

9For  the  purpose  of  this  report,  we  define  safeguards  to  include  accountability 
mechanisms.  We  note  that  we  previously  identified  an  independent  and  credible  employee 
appeals  mechanism  as  a  safeguard.  For  example,  see  GAO-06-582R.  However,  although  the 
National  Defense  Authorization  Act  for  Fiscal  Year  2004  (Pub.  L.  No.  108-136,  §  1101(a), 
(2003))  gave  DOD  the  authority  to  establish  a  different  process  for  employees  to  appeal 
adverse  actions  than  the  appeals  process  available  to  employees  under  Title  5,  that  DOD 
authority  was  rescinded  by  the  National  Defense  Authorization  Act  for  Fiscal  Year  2008, 
Pub.  L.  No.  110-181,  §  1106(a),  (2008).  This  rescission  of  authority  gave  civilian  employees 
under  NSPS  the  same  right  to  appeal  adverse  actions  to  the  Merit  Systems  Protection 
Board  that  they  had  under  Title  5.  5  U.S.C.  §§  7513(d),  7701. 


Page  3 


GAO-10-102  Human  Capital 


•  Provide  adequate  training  and  retraining  for  supervisors,  managers, 
and  employees  in  the  implementation  and  operation  of  the 
performance  management  system. 

•  Institute  a  process  for  ensuring  ongoing  performance  feedback  and 
dialogue  between  supervisors,  managers,  and  employees  throughout 
the  appraisal  period,  and  setting  timetables  for  review. 

•  Assure  that  certain  predecisional  internal  safeguards  exist  to  help 
achieve  consistency,  equity,  nondiscrimination,  and  nonpoliticization 
of  the  performance  management  process  (e.g.,  independent 
reasonableness  reviews  by  a  third  party  or  reviews  of  performance 
rating  decisions,  pay  determinations,  and  promotions  before  they  are 
finalized  to  ensure  that  they  are  merit-based,  as  well  as  pay  panels  who 
consider  the  results  of  the  performance  appraisal  process  and  other 
information  in  connection  with  final  pay  decisions). 

•  Assure  that  there  are  reasonable  transparency  and  appropriate 
accountability  mechanisms  in  connection  with  the  results  of  the 
performance  management  process,  including  periodic  reports  on 
internal  assessments  and  employee  survey  results  relating  to 
performance  management  and  individual  pay  decisions  while 
protecting  individual  confidentiality. 

•  Assure  that  the  agency’s  performance  management  system  results  in 
meaningful  distinctions  in  individual  employee  performance. 

•  Provide  a  means  for  ensuring  that  adequate  agency  resources  are 
allocated  for  the  design,  implementation,  and  administration  of  the 
performance  management  system. 

We  initially  developed  a  list  of  safeguards  based  on  our  extensive  body  of 
work  looking  at  the  performance  management  practices  used  by  leading 
public  sector  organizations  both  in  the  United  States  and  in  other 
countries.  These  safeguards  were  then  merged  with  the  list  of  safeguards 
specified  in  section  9902(b)(7)  of  Title  5,  U.S.  Code,  to  develop  the 
aforementioned  safeguards.  It  is  important  to  keep  in  mind  that  the  list  of 
safeguards  we  are  using  to  assess  DOD’s  efforts  is  not  intended  to  cover 
all  attributes  of  a  modern,  results-oriented  performance  management 
system;  therefore,  other  safeguards  may  exist.  Rather,  as  we  previously 
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reported,  the  items  on  the  list  cover  possible  safeguards  that  help  ensure 
those  systems  are  fair,  effective,  and  credible.10 

In  responding  to  the  congressional  mandate  for  the  2008  review  of  NSPS, 
we  reported,  in  September  2008,  that  while  DOD  has  taken  some  steps  to 
ensure  that  NSPS  is  fair,  effective,  and  credible,  implementation  of  certain 
safeguards  could  be  improved.  We  further  reported  that  although  DOD 
civilian  employees  under  NSPS  responded  positively  regarding  some 
aspects  of  performance  management,  the  department  does  not  have  an 
action  plan  to  address  employees’  negative  perceptions  of  NSPS.  To 
address  our  findings,  we  recommended  that  DOD  improve  the 
implementation  of  certain  safeguards  and  develop  and  implement  an 
action  plan  to  address  employees’  concerns  about  NSPS.  DOD  generally 
concurred  with  our  recommendations,  with  the  exception  of  one — 
requiring  a  predecisional  analysis  of  its  rating  results.  We  discuss  the  steps 
DOD  has  taken  to  address  our  previous  recommendations  later  in  this 
report. 

To  address  the  congressional  mandate  for  the  2009  review  of  NSPS,  we 
established  the  following  objectives:  (1)  To  what  extent  has  DOD 
implemented  safeguards  to  ensure  the  fairness,  effectiveness,  and 
credibility  of  the  NSPS  performance  management  system  and  monitored 
the  implementation  of  the  safeguards?  (2)  How  do  DOD  civilian  personnel 
perceive  NSPS,  and  what  actions  has  DOD  taken  to  address  these 
perceptions? 

For  our  first  objective,  we  reviewed  the  safeguards  specified  in  section 
9902(b)(7)  of  Title  5,  U.S.  Code,  for  the  NSPS  performance  management 
system,  as  well  as  other  safeguards  we  have  previously  identified  as  key 
for  performance  management  systems.  We  also  obtained  and  analyzed 
regulations,  guidance,  and  other  documentation  provided  by  officials  in 
DOD  and  the  NSPS  program  offices  of  the  four  components — the  Army, 
the  Air  Force,  the  Navy,  and  the  Fourth  Estate11 — and  interviewed 
knowledgeable  officials  in  all  of  these  offices  to  obtain  a  comprehensive 


10GAO-03-741T. 

uThe  Department  of  the  Navy’s  NSPS  policies  encompass  Marine  Corps  civilians.  The 
Fourth  Estate  includes  all  organizational  entities  in  DOD  that  are  not  in  the  military 
departments  or  the  combatant  commands.  Examples  of  Fourth  Estate  entities  are  the 
Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Office  of  the  DOD  Inspector  General, 
the  defense  agencies,  and  DOD  field  activities. 
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understanding  of  their  efforts  to  implement  the  NSPS  performance 
management  system  and  the  safeguards  as  well  as  the  processes, 
procedures,  and  controls  used  for  monitoring  and  overseeing 
implementation  of  the  system.  In  addition,  we  visited  eight  DOD 
organizations — two  for  each  component — located  outside  the  continental 
United  States  to  assess  the  extent  to  which  the  department  has 
implemented  safeguards  for  NSPS.  Because  our  2008  review  focused  on 
assessing  implementation  of  NSPS  and  the  safeguards  at  locations  that 
were  geographically  distributed  throughout  the  United  States,  we  focused 
our  efforts  for  this  review  on  visiting  locations  outside  the  continental 
United  States. 12  During  our  site  visits  we  reviewed  documents,  such  as  pay 
pool  business  rules,13  and  met  with  the  performance  review  authority,  pay 
pool  managers,  pay  pool  panel  members,  rating  officials,  the  NSPS 
program  manager  or  transition  team,  and  DOD  civilian  employees  under 
NSPS  to  obtain  their  perspectives  on  the  extent  to  which  the  safeguards 
had  been  implemented.  The  organizations  we  visited  were  selected  based 
on  a  number  of  factors,  such  as  the  presence  of  a  large  number  or 
concentrated  group  of  civilian  employees  under  NSPS.  For  our  second 
objective,  we  obtained  and  analyzed  the  results  of  DOD’s  May  2006, 
November  2006,  May  2007,  and  February  2008  Status  of  Forces  Survey  of 
Civilian  Employees,  which  gauge  employee  attitudes  toward  NSPS,  among 
other  things.  We  used  these  data  to  identify  indications  of  movement  or 
trends  in  employee  perceptions.  As  we  reported  in  our  first  assessment  of 
NSPS,  we  reviewed  the  results  of  prior  administrations  of  DOD’s  Status  of 
Forces  Survey  of  Civilian  Employees  and  found  the  results  to  be 
sufficiently  reliable  for  the  purposes  of  our  report.  For  our  current 
assessment,  we  found  no  substantial  changes  in  the  methodology  for  the 
2008  Status  of  Forces  Survey  of  Civilian  Employees.  In  addition,  we 
obtained  the  perspectives  of  DOD  civilian  personnel  at  the  eight  locations 
we  visited  by  conducting  small  group  discussions  with  nonsup ervisory  and 
supervisory  employees  under  NSPS  and  administering  a  short 
questionnaire.  Because  the  locations  we  visited  are  not  representative  of 
all  DOD  locations,  the  results  from  our  discussions  and  the  comments 


12 According  to  the  Program  Executive  Office,  as  of  September  2008,  about  85  percent  of  the 
civilian  employees  under  NSPS  were  located  in  the  continental  United  States,  whereas 
about  15  percent  were  located  outside  the  continental  United  States.  Of  the  civilian 
employees  under  NSPS  located  outside  the  continental  United  States,  about  54  percent 
were  in  either  Germany  or  Hawaii. 

13Business  rules  are  the  policies  that  govern  a  pay  pool’s  operations.  They  may  specify,  for 
example,  the  pay  pool  panel’s  structure,  roles  and  responsibilities,  standards  of  conduct, 
and  the  processes  used  for  reconciling  employee  ratings  and  allocating  shares  under  NSPS. 
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provided  are  not  generalizable  to  the  entire  NSPS  civilian  population. 
However,  the  composition  of  our  discussion  groups  was  designed  to 
ensure  that  we  spoke  with  employees  from  each  of  the  four  DOD 
components  at  locations  outside  the  continental  United  States.  Further 
details  about  our  scope  and  methodology  can  be  found  in  appendix  I. 

We  conducted  this  performance  audit  from  November  2008  through 
September  2009  in  accordance  with  generally  accepted  government 
auditing  standards.  Those  standards  require  that  we  plan  and  perform  the 
audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable 
basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We 
believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives. 


Results  in  Brief 


DOD  continues  to  take  steps  to  implement  safeguards  as  part  of  the  NSPS 
performance  management  system,  but  implementation  of  some  safeguards 
could  be  improved,  and  continued  monitoring  of  the  safeguards’ 
implementation  is  needed.  In  general,  DOD  has  taken  some  steps  to  meet 
the  intent  of  each  of  the  safeguards,  and  it  has  implemented  some  of  the 
recommendations  from  our  2008  report,  including  requiring  commands  to 
publish  the  final  overall  rating  results  and  providing  guidance  to  pay  pools 
encouraging  managers  to  rate  employees  appropriately.  Nonetheless, 
DOD’s  implementation  of  some  of  the  safeguards  could  be  improved.  First, 
for  example,  DOD  has  not  evaluated  the  effectiveness  of  the  training  that  it 
provides  to  employees  on  the  system.  Under  NSPS,  the  components  are 
responsible  for  training  their  employees,  while  DOD’s  NSPS  Program 
Executive  Office  (PEO)  supports  the  components’  efforts  by  offering  a 
variety  of  departmentwide  training  courses  and  other  materials.  However, 
officials14  at  each  of  the  eight  organizations  we  visited  expressed  concerns 
over  the  effectiveness  of  the  training  provided,  noting  that  additional 
training  was  needed,  that  training  was  not  always  helpful,  and  that  some 
training  was  not  timely  or  was  outdated.  We  previously  reported  that  it  is 
increasingly  important  for  agencies  to  measure  the  real  impact  of  training 
and  thus  evaluate  the  training  efforts  to  ascertain  progress  toward 


14For  the  purpose  of  this  report,  we  use  the  term  officials  to  refer  to  the  management 
officials  we  met  with  during  our  site  visits,  including  the  performance  review  authority,  pay 
pool  manager,  pay  pool  panel  members,  rating  officials,  and  NSPS  transition  manager  or 
members  of  the  NSPS  transition  team. 
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achieving  agency  goals.16  Second,  while,  in  2008,  DOD  did  not  agree  with 
our  recommendation  to  require  a  predecisional  analysis  of  ratings — stating 
that  postdecisional  analysis  of  rating  results  is  useful  to  identify  barriers 
and  corrective  actions — we  found  during  the  course  of  this  review  that  the 
department’s  postdecisional  analyses  following  the  2007  and  2008  NSPS 
performance  management  cycles  lacked  consistency  and  did  not  generally 
include  demographic  information.  However,  in  May  2009,  the  PEO  issued 
guidance  to  promote  a  degree  of  standardization  in  the  components’ 
postdecisional  analyses.  While  not  predecisional,  we  recognize  that  DOD’s 
approach  represents  a  noteworthy  step  and  does  provide  some  benefits, 
some  of  which  are  similar  to  those  of  a  predecisional  analysis.  For 
example,  like  predecisional  analysis,  postdecisional  analysis  is  a 
mechanism  to  ensure  that  employees  receive  fair  and  equitable  treatment 
in  all  aspects  of  personnel  management. 16  However,  the  guidance  does  not 
specify  what  process  the  components  should  follow  to  investigate 
potential  barriers  to  fair  and  equitable  ratings  and  their  causes,  nor  a 
process  for  eliminating  barriers  that  are  found.  Because  DOD’s  guidance 
does  not  specify  these  steps,  the  components  may  not  follow  a  consistent 
approach  when  investigating  potential  barriers,  which  could  hinder  their 
efforts  to  eliminate  them.  In  our  2008  report,  we  also  noted  that  continued 
monitoring  of  the  safeguards  was  needed  to  ensure  that  DOD’s  actions 
were  effective  as  implementation  of  NSPS  proceeded.  We  found  that  DOD 
monitors  some  aspects  of  the  implementation  of  NSPS,  such  as  whether 
the  system  is  on  track  to  achieve  certain  goals  it  established,  but  does  not 
monitor  how  or  the  extent  to  which  the  safeguards  specifically  are 
implemented  across  the  department.  Because  DOD  does  not  monitor  the 
implementation  of  the  safeguards,  decision  makers  in  DOD  and  the 
Congress  lack  information  that  could  be  used  to  determine  whether  the 
department’s  actions  are  effective  and  whether  the  system  is  being 
implemented  in  a  fair,  equitable,  and  credible  manner.  Accordingly,  until 
DOD  effectively  implements  the  safeguards  and  monitors  their 
implementation  across  the  department,  employees  will  not  have  assurance 
that  NSPS  is  a  fair,  equitable,  and  credible  system  and  decision  makers  in 


15GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development 
Efforts  in  the  Federal  Government,  GAO-04-546G  (Washington,  D.C.:  March  2004). 

16Specifically,  5  U.S.C.  §  2301(b)(2)  provides  that  “all  employees  and  applicants  for 
employment  should  receive  fair  and  equitable  treatment  in  all  aspects  of  personnel 
management  without  regard  to  political  affiliation,  race,  color,  religion,  national  origin,  sex, 
marital  status,  age,  or  handicapping  condition,  and  with  proper  regard  for  their  privacy  and 
constitutional  rights.” 
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DOD  and  the  Congress  will  not  have  the  information  that  could  be  used  to 
determine  if  the  department’s  actions  are  effective. 

DOD  civilian  personnel  have  mixed  perceptions  about  NSPS,  and  while  the 
department  has  taken  some  steps  toward  addressing  their  concerns,  it  has 
not  yet  developed  and  implemented  an  action  plan  to  address  areas  where 
employees  express  negative  perceptions  of  the  system,  as  we 
recommended  in  2008.  DOD’s  survey  data  from  2008  revealed  that  overall, 
NSPS  employees  responded  positively  about  some  aspects  of  performance 
management,  such  as  connecting  pay  to  performance,  and  negatively 
about  other  aspects,  including  the  performance  appraisal  process.  For 
example,  DOD’s  survey  data  for  2008  indicate  that  an  estimated  42  percent 
of  NSPS  employees  agree  that  pay  raises  depend  on  how  well  employees 
perform  their  jobs,  as  compared  with  an  estimated  25  percent  of  non-NSPS 
employees.  However,  when  asked  about  their  performance  appraisal 
system,  an  estimated  29  percent  of  NSPS  employees,  as  compared  with  an 
estimated  34  percent  of  non-NSPS  employees,  agreed  with  the  statement 
that  their  performance  appraisal  system  improves  organizational 
performance.  Additionally,  DOD’s  most  recent  survey  data  indicated  that 
employees  who  worked  under  NSPS  the  longest — spiral  1.1  employees17 — 
expressed  negative  perceptions  of  the  system  consistent  with  those  we 
reported  on  in  2008.  For  example,  between  the  November  2006  and 
February  2008  administrations  of  DOD’s  survey,  the  percentage  of  spiral 
1.1  employees  that  agreed  that  they  understood  what  they  had  to  do  to  be 
rated  at  a  different  performance  level  declined  from  an  estimated  59 
percent  in  November  2006  to  an  estimated  53  percent  in  May  2007,  then 
remained  consistent  with  an  estimated  54  percent  responding  that  they 
agree  in  February  2008.  Further,  the  results  of  our  discussion  groups 
indicated  that  employees  and  supervisors  continue  to  have  consistent 
concerns  and  negative  perceptions  of  NSPS.  These  included  the  following: 
(1)  NSPS’s  negative  impact  on  employee  motivation  and  morale;  (2)  the 
excessive  amount  of  time  spent  navigating  the  performance  management 
process;  (3)  challenges  with  job  objectives;  (4)  factors  undermining 
employee  confidence  in  the  system,  such  as  the  subjectivity  of  the  pay 
pool  panel  process;  and  (5)  factors  unrelated  to  job  performance  affecting 
employees’  final  performance  ratings,  such  as  supervisors’  and  employees’ 
writing  skills.  As  we  have  previously  reported,  these  negative  perceptions 


1  DOD  used  a  phased  approach  to  converting  civilian  employees  to  NSPS.  Each  phase  is 
called  a  spiral  and  each  spiral  has  an  identifying  number  associated  with  it — for  example, 
spiral  1.1. 
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are  not  surprising  given  that  large-scale  organizational  transformations — 
for  example,  the  adoption  of  a  new  performance  management  system — 
often  entail  fundamental  and  radical  changes  that  require  an  adjustment 
period  to  gain  employees’  trust  and  acceptance.  With  regard  to  DOD’s 
efforts  to  address  employee  perceptions  of  the  system,  DOD  issued  a 
memorandum,  in  June  2009,  highlighting  actions  the  department  had  taken 
in  this  area,  as  well  as  suggested  future  actions  that  the  components  could 
take  to  address  identified  negative  perceptions.  While  we  believe  that 
issuing  this  memorandum  is  an  important  first  step  in  addressing 
employee  concerns,  it  does  not  fully  meet  the  intent  of  our  2008 
recommendation  that  DOD  develop  and  implement  a  specific  action  plan 
to  address  employee  perceptions  of  NSPS  because  it  does  not  specify  such 
things  as  the  actions  DOD  intends  to  take,  who  will  be  responsible  for 
taking  the  action,  and  timelines  for  doing  so — items  stipulated  in  OPM 
guidance  on  action  planning.18  We  continue  to  believe  that  implementing 
such  an  action  plan  is  important,  and  note  that  doing  so  would  be  a  step 
that  DOD  could  take  to  involve  employees  in  the  system’s 
implementation — which  is  one  of  the  safeguards  we  discuss. 

In  September  2009,  we  provided  DOD  with  a  draft  of  this  report  that 
included  three  recommendations  to  better  address  the  safeguards  and 
improve  implementation  of  the  NSPS  performance  management  system. 
Subsequently,  as  we  were  preparing  to  issue  this  report,  the  proposed 
National  Defense  Authorization  Act  for  Fiscal  Year  2010  passed  both 
Houses  of  Congress.19  That  legislation  contained  provisions  that,  if 
enacted,  would  terminate  NSPS  while  also  requiring  that  DOD  include 
certain  safeguards  as  part  of  any  future  performance  management  system. 
Citing  the  uncertainty  regarding  the  system's  future,  DOD  partially 
concurred  with  our  three  recommendations  specific  to  the  NSPS 
performance  management  system.  DOD  also  stated  that  it  would  consider 
acting  on  our  recommendations  to  the  extent  they  are  relevant  as  the 
department  moves  forward  with  any  future  performance  management 
system.  We  believe  that  this  is  a  reasonable  approach.  Accordingly,  we 
have  revised  our  recommendations  to  apply  to  any  future  performance 
management  system  for  DOD’s  civilian  employees — whether  NSPS  or 
another  system.  Specifically,  we  are  recommending  that  DOD  (1)  review 


18Office  of  Personnel  Management  Memorandum,  “Guidance  on  Employee  Satisfaction 
Action  Planning”  (Aug.  19,  2009). 

19H.R.  Conf.  Rep.  No.  111-288,  §1113  (Oct.  7,  2009)  to  accompany  H.R.  2647,  the  proposed 
National  Defense  Authorization  Act  for  2010. 
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and  evaluate  the  effectiveness  of  the  department’s  performance 
management  training;  (2)  ensure  that  guidance  is  in  place  for  conducting  a 
postdecisional  analysis  that  specifies  what  process  the  components  should 
follow  to  investigate  and  eliminate  potential  barriers  to  fair  and  equitable 
ratings;  and  (3)  include,  as  part  of  the  department’s  monitoring  of  the 
implementation  of  its  system,  efforts  to  monitor  and  evaluate  how  the 
safeguards  specifically  are  being  implemented  by  lower-level  organizations 
across  the  department.  DOD’s  comments  are  reprinted  in  appendix  III. 


Background 


The  General  Schedule  (GS)  is  the  federal  government’s  primary  pay  and 
classification  system  for  white-collar  employees.  Under  this  system, 
federal  employees  are  paid  according  to  governmentwide  rules  and 
procedures,  and  federal  positions  are  classified  according  to  the  difficulty 
and  responsibility  of  the  work  performed.  The  GS  system  was  created  in 
1949,  when  most  federal  positions  involved  clerical  work  or  revolved 
around  the  execution  of  established,  stable  processes — for  example, 
posting  census  figures  in  ledgers  or  retrieving  taxpayer  records  from  vast 
file  rooms.  Over  time,  however,  federal  positions  have  become 
increasingly  specialized  and  more  highly  skilled.  In  light  of  this  change,  a 
number  of  federal  agencies  have  attempted  to  provide  managers  with 
greater  flexibility  in  hiring  and  awarding  pay  raises  to  employees  by 
implementing  human  capital  initiatives,  such  as  performance  management 
systems,  that  reward  employees’  performance  and  contribution  to  the 
agency’s  mission.  The  need  for  human  capital  reform  regarding  these 
systems  has  been  the  subject  of  a  number  of  previous  GAO  reviews.20 
These  reviews  have  noted,  for  example,  that  federal  agencies  must  have 
modern,  effective,  credible,  and  validated  performance  management 
systems  that  are  capable  of  supporting  pay  and  other  personnel  decisions, 
and  that  pay  for  performance  works  only  with  adequate  safeguards  and 
appropriate  accountability  mechanisms  in  place  to  ensure  that  the 
safeguards  are  implemented  in  a  fair,  effective,  and  credible  manner. 


20See  GAO,  Results- Oriented  Management:  Opportunities  Exist  for  Refining  the 
Oversight  and  Implementation  of  the  Senior  Executive  Performance-Based  Pay  System, 
GAO-09-82  (Washington,  D.C.:  Nov.  21,  2008);  Office  of  Personnel  Management:  Key 
Lessons  Learned  to  Date  for  Strengthening  Capacity  to  Lead  and  Implement  Human 
Capital  Reforms,  GAO-07-90  (Washington,  D.C.:  Jan.  19,  2007);  Residts-Oriented  Cultures: 
Modern  Performance  Management  Systems  Are  Needed  to  Effectively  Support  Pay  for 
Performance,  GAO-03-612T  (Washington,  D.C.:  Apr.  1,  2003);  and  GAO-03-741T. 
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In  November  2003,  the  Congress  included  a  provision  in  the  National 
Defense  Authorization  Act  for  Fiscal  Year  2004  providing  DOD  with  the 
authority  to  establish  a  pay-for-performance  management  system  as  part 
of  NSPS.21  In  April  2004,  the  Secretary  of  Defense  appointed  an  NSPS 
Senior  Executive  to,  among  other  things,  design,  develop,  and  implement 
NSPS.  Under  the  Senior  Executive’s  authority,  the  PEO  was  established  as 
the  central  policy  and  program  office  for  NSPS.  The  PEO’s  responsibilities 
include  designing  the  human  resource/pay-for-performance  systems, 
developing  communication  and  training  strategies,  modifying  personnel 
information  technology,  and  preparing  joint  enabling  regulations  and 
internal  DOD  implementing  regulations,  called  implementing  issuances.  As 
the  central  DOD-wide  program  office,  the  PEO  also  directs  and  oversees 
the  four  components’  NSPS  program  managers,  who  report  to  their  parent 
components  and  the  PEO.  These  program  managers  serve  as  their 
components’  NSPS  action  officers  and  also  participate  in  the  development, 
planning,  implementation,  and  deployment  of  NSPS. 

Beginning  in  April  2006,  DOD  began  phasing  (or  spiraling)  civilian 
employees  into  NSPS;22  however,  in  January  2008  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2008  prohibited  the  Secretary  of  Defense 
from  converting  more  than  100,000  employees  to  NSPS  in  any  calendar 
year  and  excluded  Federal  Wage  System  employees  from  coverage  under 
NSPS.23  Further,  in  March  2009,  DOD  announced  that  it  would  delay  the 
conversion  of  additional  organizations  to  NSPS  pending  the  outcome  of  a 
joint  review  of  the  system  by  DOD  and  OPM.  According  to  DOD,  this 
decision  affected  roughly  2,000  employees  in  organizations  scheduled  to 
convert  to  NSPS  during  the  spring  of  2009.  As  a  result  of  these  and  other 
legislative  changes  that  resulted  in  revisions  to  the  NSPS  regulations,  the 
PEO  has  not  developed  new  timelines  for  phasing  any  additional  civilian 
employees  into  NSPS.  As  table  1  shows,  according  to  DOD,  almost  220,000 
civilian  employees  have  been  phased  into  NSPS  as  of  September  2009. 


21Pub.  L.  No.  108-136,  §  1101  (2003)  (codified  at  5  U.S.C.  §§  9901-9904).  The  National 
Defense  Authorization  Act  for  Fiscal  Year  2008  amended  5  U.S.C.  §  9902.  Pub.  L.  No.  110- 
181,  §  1106  (2008). 

22 According  to  PEO  officials,  DOD  originally  planned  to  convert  approximately  700,000 
civilian  employees  to  NSPS;  however,  legislative  changes  have  decreased  the  number  of 
civilian  employees  eligible  to  convert  to  the  system. 

23Pub.  L.  No.  110-181,  §  1106(a)  (2008).  The  Federal  Wage  System  is  a  uniform  pay-setting 
system  that  covers  federal  appropriated  fund  and  nonappropriated  fund  blue-collar 
employees  who  are  paid  by  the  hour. 
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Table  1 :  Number  of  DOD  Civilian  Employees  Phased  into  NSPS,  as  of  September  2009 


Spiral 

Army 

Air  Force 

Navy 

Fourth  Estate 

Total  number 
of  employees 

1.1 

2,666 

3,187 

4,828 

1,611 

12,292 

1.2 

18,084 

38,693 

10,087 

9,239 

76,103 

1.3 

28,582 

1,102 

8,559 

391 

38,634 

2.1 

11,103 

- 

8,194 

85 

19,382 

2.2 

20,175 

- 

22,487 

15,145 

57,807 

2.3 

483 

- 

14,128 

38 

14,649 

2.4 

- 

- 

294 

15 

309 

Total  number  of  employees 

81,093 

42,982 

68,577 

26,524 

219,176 

Source:  DOD. 

Note:  In  J  une  2009,  we  reported  that  according  to  DOD,  about  212,000  civilian  employees  were 
under  NSPS.  See  GAO,  Human  Capital:  Continued  Monitoring  of  Internal  Safeguards  and  an  Action 
Plan  to  Address  Employee  Concerns  Could  Improve  Implementation  of  the  National  Security 
Personnel  System,  GAO-09-840  (Washington,  D.C.:J  une  25,  2009).  According  to  a  PEO  official,  no 
additional  organizations  have  converted  to  NSPS  since  March  2009  and  the  growth  of  the  NSPS 
workforce  is  due  to  routine  hiring,  transfer,  reassignment,  and  promotion  actions  into  organizations 
already  under  NSPS. 

The  NSPS  performance  management  process  is  ongoing  and  consists  of 
several  phases  that  are  repeated  during  each  annual  performance  cycle. 
The  process  begins  with  a  planning  phase  that  involves  supervisors  (or 
rating  officials)  and  employees  working  together  to  establish  performance 
plans.  This  includes  developing  job  objectives — the  critical  work 
employees  perform  that  is  aligned  with  their  organizational  goals  and 
focused  on  results — and  identifying  contributing  factors — the  attributes 
and  behaviors  that  identify  how  the  critical  work  established  in  the  job 
objectives  is  going  to  be  accomplished  (e.g.,  cooperation  and  teamwork). 
After  the  planning  phase  comes  the  monitoring  and  developing  phase, 
during  which  ongoing  communication  between  supervisors  and  employees 
occurs  to  ensure  that  work  is  accomplished;  attention  is  given  to  areas 
that  need  to  be  addressed;  and  managers,  supervisors,  and  employees  have 
a  continued  and  shared  understanding  of  expectations  and  results.  In  the 
rating  phase,  the  supervisor  prepares  a  written  assessment  that  captures 
the  employee’s  accomplishments  during  the  appraisal  period.  In  the 
final — or  reward — phase,  employees  should  be  appropriately  rewarded  or 
compensated  for  their  performance  with  performance  payouts.  During  this 
phase,  employee  assessments  are  reviewed  by  multiple  parties  to 
determine  employees’  ratings  and,  ultimately,  performance  payouts. 
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The  performance  management  process  under  NSPS  is  organized  by  pay 
pools.  A  pay  pool  is  a  group  of  employees  who  share  in  the  distribution  of 
a  common  pay-for-performance  fund.24  The  key  participants  that  make  up 
pay  pools  are  the  employee,  supervisor,  higher-level  reviewer,  pay  pool 
panel,  pay  pool  manager,  performance  review  authority,  and,  in  some 
instances,  the  sub-pay  pool,25  as  shown  in  figure  1. 


24Criteria  to  distinguish  pay  pools  may  include,  but  are  not  limited  to,  organization 
structure,  employee  job  function,  location,  and  organization  mission. 

25Where  determined  appropriate  because  of  the  size  of  the  pay  pool  population,  the 
complexity  of  the  mission,  the  need  to  prevent  conflicts  of  interest,  or  other  similar  criteria, 
sub-pay  pool  panels  may  be  organized  in  a  structure  subordinate  to  the  pay  pool  panel. 
Sub-pay  pool  panels  normally  operate  under  the  same  requirements  and  guidelines 
provided  to  the  pay  pool  panels  to  which  they  belong. 
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Figure  1 :  Example  of  NSPS  Pay  Pool  Organization 
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Source:  DOD. 


Within  a  pay  pool,  each  participant  has  defined  responsibilities  under  the 
performance  management  process: 

•  Employees  are  encouraged  to  be  involved  throughout  the  performance 
management  cycle.  This  includes  initially  working  with  their 
supervisors  to  develop  job  objectives  and  identify  associated 
contributing  factors;  identifying  and  recording  accomplishments  and 
results  throughout  the  appraisal  period;  and  participating  in  interim 
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reviews  and  end-of-year  assessments,  for  example,  by  preparing  a  self- 
assessment  of  their  performance  during  the  annual  appraisal  period. 

•  Supervisors  (or  rating  officials)  are  responsible  for  effectively 
managing  the  performance  of  their  employees.  This  includes  clearly 
communicating  performance  expectations;  aligning  performance 
expectations  and  employee  development  with  organization  mission 
and  goals;  working  with  employees  to  develop  written  job  objectives 
that  reflect  expected  accomplishments  and  contributions  for  the 
appraisal  period  and  identifying  applicable  contributing  factors; 
providing  employees  meaningful,  constructive,  and  candid  feedback 
relative  to  performance  expectations,  including  at  least  one 
documented  interim  review;  making  meaningful  distinctions  among 
employees  based  on  performance  and  contribution;  and  providing 
recommended  ratings  of  record,  share  assignments,  and  payout 
distributions  to  the  pay  pool. 

•  The  higher-level  reviewer,  typically  the  rating  official’s  supervisor,  is 
responsible  for  reviewing  and  approving  job  objectives  and 
recommended  employee  assessments.  The  higher-level  reviewer  is  the 
first  step  in  ensuring  consistency  of  ratings  because  this  individual 
looks  across  multiple  ratings. 

•  The  pay  pool  panel  (or,  in  some  cases,  the  sub-pay  pool  panel)  is  a 
board  of  management  officials  who  are  usually  in  positions  of  line 
authority  or  in  senior  staff  positions  with  resource  oversight  for  the 
organizations,  groups,  or  categories  of  employees  making  up  the  pay 
pool  membership.26  The  primary  function  of  the  pay  pool  panel  is  the 
reconciliation  of  ratings  of  record,  share  distribution,  and  payout 
allocation  decisions.  For  example,  the  pay  pool  panel  may  adjust  a 
supervisor’s  recommended  rating  of  record  in  order  to  help  ensure  the 
equity  and  consistency  of  ratings  across  the  pay  pool. 

•  Each  pay  pool  has  a  manager  who  is  responsible  for  providing 
oversight  of  the  pay  pool  panel.  The  pay  pool  manager  is  the  final 
approving  official  of  the  rating  of  record.  Performance  payout 


26Pay  pool  panel  members  may  not  participate  in  payout  deliberations  or  decisions  that 
directly  affect  their  own  performance  assessments  or  pay. 
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determinations  may  be  subject  to  higher  management  review  by  the 
performance  review  authority27  or  equivalent  review  process. 

•  Finally,  the  performance  review  authority  provides  oversight  of  several 
pay  pools  and  addresses  the  consistency  of  performance  management 
policies  within  a  component,  major  command,  field  activity,  or  other 
organization  as  determined  by  the  component. 


DOD  Continues  to 
Make  Progress,  but 
Implementation  of 
Some  Safeguards 
Could  Be  Improved, 
and  Continued 
Monitoring  of  the 
System’s 
Implementation, 
Including  the 
Safeguards,  Is  Needed 


DOD  continues  to  take  some  steps  to  implement  each  of  the  safeguards  we 
reported  on  in  September  2008.  However,  opportunities  exist  to 
continually  involve  employees  in  the  system’s  implementation,  and 
implementation  of  some  safeguards — for  example,  providing  effective 
training — could  be  improved.  Also,  we  previously  reported  that  continued 
monitoring  of  the  safeguards  was  needed  to  help  ensure  that  DOD’s  efforts 
were  effective  as  implementation  of  NSPS  proceeded.  However,  we  found 
that  while  DOD  monitors  some  aspects  of  the  implementation  of  NSPS,  it 
does  not  monitor  how  the  safeguards  specifically  are  implemented  across 
the  department.  Because  DOD  does  not  monitor  the  safeguards’ 
implementation,  decision  makers  in  DOD  lack  information  that  could  be 
used  to  determine  whether  the  department’s  actions  are  effective  and 
whether  the  system  is  being  implemented  in  a  fair,  equitable,  and  credible 
manner. 


2lThe  senior  organization  official,  usually  a  member  of  the  Senior  Executive  Service  or  a 
General/Flag  officer,  serves  as  the  performance  review  authority.  DOD  components  may 
provide  additional  guidance  for  the  establishment  of  performance  review  authorities.  The 
responsibilities  of  the  performance  review  authority  may  be  assigned  to  an  individual 
management  official  or  organizational  unit  or  group. 
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Implementation  of  Some 
Safeguards  Could  Be 
Improved 

Involve  Employees  in  the 
Design  and  Implementation  of 
the  System 


DOD  has  taken  a  number  of  steps  to  involve  employees  and  stakeholders 
in  the  design  and  implementation  of  NSPS.  In  our  September  2008  report, 
we  noted  that  DOD  solicited  comments  from  employees  and  unions 
representing  DOD  employees  during  the  design  of  NSPS.  We  also  noted 
that  DOD  involved  employees  during  the  implementation  of  the  system  in 
workshops  and  other  efforts  aimed  at  refining  the  system’s  performance 
factors.  DOD  continues  to  take  such  steps.  For  example: 

•  DOD  solicited  comments  from  employees  and  unions  on  the  system’s 
final  rule,28  which  was  published  in  the  Federal  Register  in  September 
2008.  According  to  the  Federal  Register,  the  final  regulations,  which 
became  effective  in  November  2008,  include  revisions  based  on  526 
comments  submitted  during  the  public  comment  period  and  on 
comments  from  9  of  the  department’s  10  unions  with  national 
consultation  rights. 

•  DOD  involved  employees  in  efforts  to  improve  the  usability  of  the 
automated  tools  that  support  the  NSPS  performance  and  pay  pool 
management  processes.  Specifically,  the  PEO  and  the  department’s 
Civilian  Personnel  Management  Service  held  a  series  of  meetings  with 
employees,  rating  officials,  pay  pool  managers,  and  human  resource 
practitioners  in  early  2008  to  address  concerns  regarding  the  usability 
of  the  automated  tools.  These  meetings  allowed  the  department  to 
gather  requirements  for  the  next  version  of  the  NSPS  automated  tools 
based  on  lessons  learned  and  user  input.  Subsequently,  DOD 
established  six  separate  working  groups  to  develop  and  evaluate  the 
requirements  for  each  of  its  automated  tools.  In  addition,  DOD 
initiated  separate  efforts  to  enhance  the  usability  of  the  Performance 
Appraisal  Application — the  DOD-wide  tool  for  employee  self- 
assessments  and  appraisals.  Specifically,  the  contractor  that  developed 
the  Performance  Appraisal  Application  enlisted  the  assistance  of 
software  usability  experts  to  evaluate  the  tool  and  recommend 
changes  that  would  enhance  users’  experience  with  it.  As  a  part  of  this 
effort,  the  contractor  observed  and  worked  with  employees  and  rating 
officials  to  identify  changes  that  could  be  made  to  the  Performance 
Appraisal  Application  to  make  it  more  user-friendly.  DOD  also  tested 


285.  C.F.R.  §  9901  (2008). 
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the  functionality  and  usability  of  the  enhancements  that  were  made  to 
the  Performance  Appraisal  Application  with  over  300  users. 

•  DOD  has  taken  steps  to  involve  the  components  in  the  implementation 
of  NSPS  through  biweekly  conference  calls  held  at  key  phases  of  the 
performance  management  process.  According  to  the  PEO,  during  these 
calls,  PEO  and  Civilian  Personnel  Management  Service  representatives 
discuss  topics  submitted  by  the  components,  respond  to  questions 
regarding  such  things  as  NSPS  policy  and  the  system’s  automated 
tools,  and  share  lessons  learned  with  participants.  Further,  according 
to  the  PEO,  these  conference  calls  allow  participants  to  address 
systemic  problems  through  feedback  shared  between  different  levels 
of  the  organization. 

At  the  locations  we  visited  outside  the  continental  United  States,  we  found 
that  some  steps  were  generally  being  taken  to  involve  employees  in  the 
implementation  of  NSPS  as  well.  For  example,  officials  at  each  of  the  eight 
locations  reported  that  organizations  identified  lessons  learned  that  were 
generally  based  upon  employee  feedback  gathered  following  the  mock  pay 
pool.29  According  to  these  officials,  lessons  learned  were  used  to  make 
changes  to,  among  other  things,  training  materials,  business  rules,  and  the 
use  of  job  objectives  and  contributing  factors.  For  example,  two  locations 
limited  the  number  of  contributing  factors  employees  should  use  in  their 
performance  plans  based  upon  lessons  learned,  while  one  location 
responded  to  employee  feedback  regarding  a  lack  of  time  to  devote  to 
NSPS  by  mandating  that  employees  be  allowed  to  take  time  to  complete 
NSPS  training. 

While  DOD  has  taken  a  number  of  steps  to  involve  employees  in  the  design 
and  implementation  of  NSPS  thus  far,  as  stated  above,  we  note  that  one 
way  the  department  could  continue  to  involve  employees  as 
implementation  of  the  system  proceeds  is  to  develop  and  implement  an 
action  plan  to  address  employees’  perceptions  of  NSPS,  as  we 
recommended  in  our  first  assessment  of  NSPS.  However,  DOD  has  not  yet 
done  so,  which  we  discuss  further  in  the  second  objective  of  this  report. 


29The  mock  pay  pool  is  a  way  for  organizations  to  understand  the  pay  pool  process.  During 
the  exercise,  organizations  identify  ways  to  improve  their  pay  pool  process  to  achieve 
greater  consistency  and  ensure  fairness  in  ratings  and  payouts.  Rating  officials  review  their 
employees  and  assign  mock  ratings,  numbers  of  shares,  and  payout  distributions.  The  panel 
then  practices  advocating  for  employees  and  reconciling  results. 
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Link  Employee  Objectives  to 
the  Agency’s  Strategic  Goals 
and  Mission 


Training  and  Retraining  in  the 
System’s  Implementation  and 
Operation 


DOD  continues  to  take  steps  to  link  employee  job  objectives  to  the 
agency’s  strategic  goals,  mission,  and  desired  outcomes.  As  we  noted  in 
our  2008  report,  DOD’s  automated  tool  for  employee  self-assessments  and 
appraisals — the  Performance  Appraisal  Application — provides  a 
designated  area  for  the  mission  of  the  employee’s  command  to  be  inserted 
as  a  guide  while  employees  compose  their  job  objectives  and  self- 
assessments.  In  May  2009,  DOD  published  its  evaluation  of  NSPS  for  2008, 
entitled  National  Security  Personnel  System  (NSPS)  -  2008  Evaluation 
Report,  which  included  an  evaluation  of  employee  performance  plans  to 
determine  the  extent  to  which  employee  job  objectives  are  aligned  with 
higher-level  organizational  goals  and  thus  ensure  that  employee 
performance  contributes  to  the  achievement  of  organizational  success. 

The  evaluation  included  240  employee  performance  plans  encompassing  a 
range  of  job  series,  pay  schedules,  pay  bands,  and  organizations  within 
each  of  the  four  components.  The  evaluation  found  that  95  percent  of  the 
objectives  were  strongly  aligned  to  higher-level  goals  and  demonstrated  a 
clear,  direct,  and  strong  linkage  to  the  organizational  mission  or  relevant 
strategic  goals. 

During  our  site  visits,  we  found  that  the  organizations  were  taking  steps  to 
ensure  that  employees  understood  how  their  job  objectives  link  to  the 
organization’s  strategic  goals.  This  was  generally  accomplished  through 
documentation  requirements  in  the  Performance  Appraisal  Application 
and  reinforced  during  employees’  discussions  with  their  supervisors.  Some 
organizations  have  taken  additional  steps  to  ensure  that  employee  job 
objectives  link  to  the  organization’s  strategic  goals.  For  example,  at  one 
location  we  visited  the  commanding  general  issued  a  memorandum  to 
managers  emphasizing  the  importance  of  ensuring  that  employee 
objectives  are  linked  to  the  command’s  mission  and  objectives  and  that 
employees  understand  their  roles  in  achieving  those  objectives.  However, 
officials  at  five  locations  also  reported  that  employees  do  not  always 
understand  this  linkage. 

DOD  continues  to  take  steps  to  provide  employees  with  required  and  other 
training  on  the  implementation  and  operation  of  the  NSPS  performance 
management  system,  but  has  not  yet  evaluated  the  effectiveness  of  the 
training  that  it  provides.  In  our  September  2008  report,  we  noted  that  DOD 
encouraged  employees  who  were  transitioning  to  NSPS  to  receive  training 
that  covered  skills  and  behaviors  necessary  to  implement  and  sustain 
NSPS;  foster  support  and  confidence  in  the  system;  and  facilitate  the 
transition  to  a  performance-based,  results-oriented  culture.  DOD  and  each 
of  the  components  continue  to  take  such  steps  to  provide  employees  with 
required  and  other  training  on  the  system,  including  introductory  training 
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for  employees  converting  to  NSPS  and  sustainment  training  for  employees 
already  under  the  system.  While  the  components  are  responsible  for 
providing  employees  with  training  on  the  NSPS  performance  management 
system,  the  PEO  supports  their  efforts  by  offering  a  variety  of 
departmentwide  training  courses  and  other  materials.  For  example,  Web- 
based  training  modules  that  the  PEO  has  developed,  such  as  its  NSPS  101 
and  iSuccess  courses  (see  fig.  2  for  sample  screen  shots),  provide 
employees  with  basic  knowledge  about  NSPS  and  performance 
management  principles  in  general,  and  are  used  by  employees  to  develop 
their  job  objectives.  As  another  example,  the  PEO  developed  training 
guides  to  educate  employees  on  changes  to  the  NSPS  classroom  materials 
resulting  from  the  revised  NSPS  regulations  and  implementing  issuances. 
The  PEO  also  developed  a  Web  site  for  accessing  NSPS  learning  materials, 
resources,  and  other  tools.  In  addition,  we  found  that  the  Air  Force  has 
begun  incorporating  training  on  NSPS  as  a  normal  part  of  its  operations 
and  is  working  to  embed  NSPS  topics  in  the  regular  training  provided  to 
Air  Force  civilians  and  servicemembers. 


Figure  2:  Sample  Screen  Shots  from  the  NSPS  101  and  iSuccess  Training  Modules 
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Rating  Performance 

The  Rate  phase  begins  after  the  performance  year  ends 
on  September  30.  Your  supervisor  evaluates  your 
performance  and  recommends  Job  objective  ratings, 
rating  of  record,  a  number  of  shares  associated  with  the 
rating  of  record,  and  payout  distribution. 

Although  your  supervisor  is  responsible  for  writing  your 
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A  higher  level  reviewer  evaluates  the  recommended 
ratings,  number  of  shares,  and  payout  distribution, 
which  are  then  submitted  to  the  pay  pool  for 
approval.  This  use  of  multiple  levels  of  reviews  ensures 
that  your  rating  and  reward  appropriately  recognize 
your  contributions. 
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Job  Objective  Reminders 

•  Job  objectives  differ  from  daily  job  tasks 
.  Good  job  objectives  afcgn  wtth  DoO's  mission  and 
organizational  goals 

.  Best  practice  is  to  create  3-5  job  objectives 
.  Performance  Indicators  have  standardized 

descriptions  to  help  identify  appropriate  levels  of 
performance 

•  Contnbuting  Factors  emphasize  behaviors  critical  to 
the  accomplishment  of  the  Job  objectives 
.  SMART  provides  an  effective  frame  work  tor  creating 
wea-witten  job  objectives  Specific.  Measurable. 
Aligned  Reakstk.  and  Timed 


- 
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Source:  DOD. 


Although  DOD  and  the  components  continue  to  take  steps  to  provide 
employees  with  training  on  NSPS,  the  department  has  not  yet  evaluated 
the  effectiveness  of  the  training  provided.  We  previously  reported  that  it  is 
increasingly  important  for  agencies  to  measure  the  real  impact  of  training 
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and  thus  evaluate  their  training — for  example,  by  establishing  clear  goals 
about  what  the  training  is  expected  to  achieve  along  with  agreed-upon 
measures  or  performance  indicators  to  ascertain  progress  toward 
achieving  those  goals.30  DOD  has  outlined  the  fundamental  requirements, 
or  goals,  of  the  training  that  it  provides  to  employees  on  NSPS. 

Specifically,  DOD  has  noted  that  for  NSPS,  a  training  program  must  be 
implemented  that  enables  employees  to  understand  better  how  to  succeed, 
and  enables  supervisors  to  communicate  performance  expectations  to 
their  employees,  provide  feedback  to  them  on  their  performance  against 
these  expectations,  and  tell  them  what  steps  they  can  take  to  improve 
their  performance  and  competencies  and  manage  their  careers.  However, 
while  DOD  has  undertaken  efforts  to  understand  employees’  perceptions 
of  its  training,  the  department  has  not  yet  evaluated  the  effectiveness  of 
the  training  that  it  provides.  For  example,  in  early  2009  the  PEO  conducted 
what  PEO  officials  describe  as  an  ad  hoc  study  of  training  needs.  The 
PEO’s  study  consisted  of  conducting  sensing  sessions  with  120  human 
resource  practitioners  at  11  locations  across  the  department.31  According 
to  the  PEO,  the  purpose  of  these  sessions  included  obtaining  feedback  on 
existing  NSPS  learning  products  and  support  and  exploring  options  for 
next  generation  products.  While  the  PEO’s  study  identified  some  needed 
improvements,  it  does  not  constitute  an  evaluation  of  the  department’s 
training — for  example,  because  it  did  not  assess  the  department’s  progress 
toward  achieving  the  goals  specified  for  the  training. 

As  another  example,  DOD’s  2008  evaluation  report  also  highlighted 
deficiencies  with  the  department’s  training  on  NSPS.32  Specifically,  the 
report  notes  that  without  effective  communication  and  training,  NSPS 
cannot  achieve  its  goal  of  being  a  credible  and  trusted  system.  Further, 
three  of  the  report’s  six  key  recommendations  focus  on  the  need  to 
enhance  the  effectiveness  of  the  training  provided  to  employees  on  NSPS: 
(1)  provide  more  training  on  the  performance  management  system;  (2) 
provide  high-level  training  for  employees  and  supervisors  that  explains  the 
pay  pool  process;  and  (3)  hold  mock  pay  pool  panels,  which  serve  as 


30GAO-04-546G. 

31Of  these  120  human  resource  practitioners,  only  1  was  located  outside  the  continental 
United  States. 

32DOD’s  2008  evaluation  focused  on  determining  whether  NSPS,  as  implemented  in  spiral  1 
organizations,  was  on  track  to  achieve  certain  goals  specified  by  DOD  and  if  any 
improvements  were  needed.  These  goals  include  whether  a  supporting  infrastructure  is  in 
place  for  the  system. 
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Ongoing  Performance 
Feedback  and  Dialogue 
between  Supervisors  and 
Employees 


refreshers  for  continuing  panel  members  and  as  training  for  new  members. 
However,  like  the  PEO’s  study,  DOD’s  2008  evaluation  report  does  not 
constitute  an  evaluation  of  the  department’s  training — for  example, 
because  it  did  not  include  an  in-depth  assessment  of  DOD’s  training  and 
also  did  not  assess  the  department’s  progress  toward  achieving  the  goals 
for  the  training,  per  agreed-upon  measures  or  performance  indicators.  Of 
the  components,  we  found  that  only  the  Army  has  taken  some  steps  to 
evaluate  the  training  it  provides  to  employees  on  the  system.  Specifically, 
the  Army  assesses  the  adequacy  of  NSPS  training  during  on-site  reviews 
that  it  conducts  as  part  of  its  implementation  of  the  system.  The  Army 
conducted  three  such  assessments  during  2008  and  an  additional  four  such 
assessments  during  2009. 

With  respect  to  our  site  visits,  although  we  found  that  each  of  the  eight 
locations  provided  training  on  NSPS  to  employees,  officials  at  each 
location  also  expressed  concerns  over  the  effectiveness  of  the  training 
provided.  For  example,  officials  at  seven  locations  told  us  that  additional 
training  was  needed  on  writing  job  objectives  or  employee  self- 
assessments  under  the  system,  while  other  officials  noted  that  additional 
training  was  needed  on  the  pay  pool  process,  use  of  the  system’s 
automated  tools,  and  how  the  reconsideration  process  works.  Similarly, 
officials  at  two  locations  raised  concerns  that  the  training  they  received 
did  not  provide  them  with  the  skills  they  needed  to  use  the  Performance 
Appraisal  Application.  For  example,  officials  told  us  that  they  received 
training  too  early  and  had  either  forgotten  it  or  the  training  had  become 
outdated  by  the  time  they  actually  used  the  Performance  Appraisal 
Application.  Further,  some  program  officials  raised  concerns  about  their 
organizations’  ability  to  provide  employees  with  adequate  training  on  the 
system  when  the  employees  are  located  outside  the  continental  United 
States.  For  example,  program  officials  at  one  location  in  Germany 
reported  challenges  providing  employees  located  in  the  field  with  training 
on  NSPS  because  they  lack  the  resources  to  send  NSPS  trainers  to  those 
locations.  However,  until  DOD  evaluates  its  training,  it  will  not  be  able  to 
determine  whether  the  training  meets  the  needs  of  its  employees  or  the 
department  is  making  progress  toward  achieving  the  goals  it  specified  for 
the  training. 

DOD  continues  to  take  steps  to  ensure  that  employees  receive  timely 
performance  feedback  that  is  meaningful,  constructive,  and  in  accordance 
with  the  department’s  requirements.  In  our  first  assessment  of  NSPS  we 
noted  that  DOD’s  implementing  issuances  required  at  least  one 
documented  interim  performance  review  and  an  annual  performance 
appraisal  and  that  the  Performance  Appraisal  Application  allowed 
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supervisors  and  employees  to  document  these  feedback  sessions.  Since 
then,  DOD  has  taken  additional  steps  to  enhance  the  Performance 
Appraisal  Application  by  modifying  the  tool  to  allow  supervisors  and 
employees  to  identify  where  they  are  in  the  performance  appraisal  process 
and  help  them  accomplish  required  actions  in  a  timely  manner. 

During  our  site  visits,  officials  at  all  eight  locations  told  us  that  NSPS  helps 
ensure  the  occurrence  of  performance  feedback  between  employees  and 
supervisors.  For  example,  officials  noted  that  use  of  the  Performance 
Appraisal  Application  encourages  employee  feedback  by  allowing 
employees  to  document  and  track  feedback  sessions,  and  that  NSPS 
encourages  direct  discussions  about  performance-related  issues,  such  as 
developing  effective  job  objectives  and  establishing  performance 
expectations. 

DOD  continues  to  take  steps  to  better  link  individual  pay  to  performance 
as  well.  As  we  noted  in  our  2008  report,  the  NSPS  performance 
management  system  uses  a  multirating  system  of  five  rating  categories — of 
which  the  lowest  rating  is  “1”  (unacceptable  performance)  and  the  highest 
rating  is  a  “5”  (role  model  performance) — that  allows  distinctions  to  be 
made  in  employee  performance  and  therefore  compensation.  Since  then, 
DOD  added  details  to  the  NSPS  regulations  to  facilitate  uniform,  equitable 
practices  across  the  department  that  accord  with  merit  system  principles. 
These  include  specifying  specific  share  assignment  ranges,  rounding  rules 
for  converting  raw  performance  scores  to  ratings,  and  formulas  for 
determining  share  value  and  the  calculation  of  performance  payouts  under 
NSPS.  DOD  also  clarified  that  a  common  share  value  should  apply 
throughout  an  entire  pay  pool.  According  to  the  PEO,  these  changes,  in 
addition  to  the  higher-level  review  of  performance  expectations, 
recommendations  for  ratings  of  record,  share  assignment,  and  payout 
distribution  have  helped  to  promote  a  more  equitable  method  for 
appraising  and  compensating  employees. 

However,  during  our  site  visits,  officials  at  seven  of  the  eight  locations  told 
us  that  they  saw  the  potential  for  factors  other  than  performance  to 
influence  employees’  ratings,  such  as  the  quality  of  employees’  and 
supervisors’  writing  skills.  For  example,  rating  officials  at  one  location 
commented  that  NSPS  does  not  reward  employees  based  on  their 
performance  but  rather  on  how  well  employees  and  supervisors  can 
communicate  in  writing  what  the  employee  achieved  during  the 
performance  management  cycle.  Similarly,  at  another  location,  a  pay  pool 
panel  member  told  us  that  the  paperwork  submitted  to  the  panel  by  both 
the  employee  and  the  supervisor  must  be  of  very  high  quality  because  it 
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can  be  difficult  to  defend  a  high  rating  recommended  for  an  employee  if 
the  assessments  are  poorly  written.  DOD’s  2008  evaluation  report  also 
highlighted  concerns  from  employees  and  managers  over  the  department’s 
success  in  linking  pay  to  performance  under  NSPS.  For  example,  DOD’s 
report  noted  that  while  some  employees  believed  that  they  saw  some  level 
of  pay  for  performance  under  NSPS,  others  were  uncertain.  Further, 

DOD’s  report  also  noted  that  some  managers  observed  that  the  quality  of 
written  assessments  contributed  significantly  to  the  way  in  which  ratings 
were  substantiated. 

We  found  that  DOD  also  continues  to  take  steps  to  ensure  that  adequate 
agency  resources  are  allocated  to  NSPS.  According  to  5  U.S.C.  § 
9902(e)(4),  to  the  maximum  extent  practicable,  for  fiscal  years  2004 
through  2012,  the  overall  amount  of  money  allocated  for  compensation  of 
civilian  employees  in  organizations  under  NSPS  shah  not  be  less  than  the 
amount  that  would  have  been  allocated  under  the  GS  system.  To  meet  the 
requirements  of  5  U.S.C.  §  9902(e)(4),  DOD’s  implementing  issuances 
require  that  the  components  certify  in  writing  to  the  Deputy  Secretary  of 
Defense  through  the  Under  Secretary  of  Defense  for  Personnel  and 
Readiness  that  the  amount  expended  for  NSPS  performance-based  pay 
increases  is  no  less  than  what  would  have  been  expended  had  these 
positions  not  been  converted  into  NSPS.  Following  the  2008  NSPS 
performance  management  cycle,  each  of  the  components  certified  that  it 
met  this  requirement. 

DOD  also  continues  to  capture  NSPS  implementation  costs,  and  for  fiscal 
year  2008,  the  PEO  reported  that  NSPS  implementation  costs  were  about 
$61.8  million.  According  to  the  PEO,  continuing  implementation  of  NSPS 
will  result  in  some  additional  program  implementation  costs,  such  as  for 
training  for  NSPS,  conducting  outreach  to  employees  and  others,  and 
improving  the  system’s  automated  tools.  However,  the  PEO  estimates  that 
once  the  conversion  of  ah  non-bargaining  unit  employees  is  complete,  the 
system’s  implementation  costs  will  decrease  significantly  unless  there  is  a 
decision  to  convert  bargaining  unit  employees.  Thereafter,  according  to 
the  PEO,  the  cost  of  administering  NSPS  will  be  no  different  than  that  of 
any  other  personnel  system. 

In  our  2008  report,  we  recommended  that  DOD  take  steps  to  better  ensure 
the  consistency  and  equity  of  the  performance  management  process  by 
requiring  a  third  party  to  perform  predecisional  demographic  and  other 
analysis  of  the  pay  pool  results.  DOD  did  not  concur  with  our 
recommendation,  stating  that  its  postdecisional  analysis  of  the  rating 
results  was  useful  for  identifying  barriers  and  any  needed  corrective 
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action.  DOD  also  stated  that  if  the  information  obtained  from  a 
postdecisional  demographic  analysis  demonstrated  that  the  results  were 
not  fair  or  equitable,  for  any  reason,  the  process  used  to  achieve  those 
results  could  be  examined  with  the  intent  to  identify  and  eliminate  barriers 
to  a  fair  and  equitable  outcome. 

However,  our  review  of  the  postdecisional  analyses  that  the  PEO  and  each 
of  the  components  completed  following  the  2007  NSPS  performance 
management  cycle  and  the  analyses  that  each  of  the  eight  organizations 
we  visited  for  our  review  completed  following  the  most  recent 
performance  management  cycle  in  2008  found  these  analyses  were 
inconsistent,  did  not  always  include  an  analysis  of  the  rating  results  by 
demographics,  and  were  generally  conducted  at  too  high  a  level  to  provide 
decision  makers  with  sufficient  information  to  identify  potential  barriers 
or  corrective  actions.  For  example,  we  found  that  following  the  2007 
performance  management  cycle,  the  PEO  analyzed  the  rating  results  for 
more  than  100,000  employees  by  select  demographic  groups,  such  as  race, 
gender,  ethnicity,  age,  veteran  status,  and  targeted  disability,  but  limited 
its  analysis  to  the  aggregate  data  from  its  pay  pools.  That  is,  the  PEO  did 
not  analyze  the  rating  results  at  the  level  where  decisions  are  made — in  the 
case  of  NSPS  ratings  and  payouts,  the  pay  pool  level.  Similarly,  in 
analyzing  the  postdecisional  analyses  that  the  components  conducted 
following  the  2007  performance  management  cycle,  we  found 
inconsistencies  in  their  approaches,  primarily  because  the  components 
were  allowed  to  develop  their  own  approaches  for  conducting  this 
analysis.  For  example,  only  the  Army  and  Fourth  Estate  included  an 
analysis  of  the  rating  results  by  demographics  as  part  of  their  respective 
postdecisional  analyses.  However,  we  also  found  that  neither  the  Army’s 
nor  the  Fourth  Estate’s  demographic  analysis  of  the  ratings  provided 
decision  makers  with  sufficient  information  to  identify  possible  barriers  or 
corrective  actions  that  could  be  taken  to  address  such  barriers.  As  with 
the  PEO,  this  problem  results  because  neither  the  Army  nor  the  Fourth 
Estate  conducted  its  analysis  at  the  pay  pool  level.  Of  the  eight  locations 
we  visited  for  our  review,  we  found  that  only  one  organization’s 
postdecisional  analysis  following  the  2008  performance  management  cycle 
included  an  analysis  of  its  ratings  results  by  demographics. 

Since  we  issued  our  2008  report,  DOD  has  taken  steps  to  promote  a  degree 
of  consistency  in  its  postdecisional  analysis  of  NSPS  ratings  and  payout 
data.  Specifically,  in  December  2008,  DOD  revised  its  implementing 
issuance  to  require  the  heads  of  DOD  components  to  conduct  an  annual 
analysis  of  NSPS  performance  ratings  and  payouts  for  subordinate 
elements,  and  issue  guidance  to  lower  echelons  and  otherwise  act  to 
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identify,  examine,  and  remove  barriers  to  similar  rating  and  payout 
potential  for  demographic  and  other  groups  in  the  workforce.  Further,  in 
May  2009,  the  PEO  issued  guidance,  entitled  Guidance  for  Conducting 
Annual  Analysis  ofNSPS  Performance  Ra  tings  and  Payouts,  in  order  to 
provide  the  components  with  a  framework  and  suggested  approaches  for 
conducting  their  annual  analysis  and  to  serve  as  a  starting  point  for 
identifying  and  examining  barriers.  For  example,  the  guidance  notes  that 
the  NSPS  performance  management  system’s  rating  and  payout  process 
has  four  main  outcomes — the  rating  of  record,  number  of  shares  awarded, 
payout,  and  the  distribution  of  the  payout — and  that  each  outcome  should 
be  reviewed.  The  guidance  also  notes  that  analysis  is  best  done  at  the  level 
where  decisions  are  made — in  the  case  of  NSPS  ratings  and  payouts,  the 
pay  pool  level.  Further,  the  guidance 

•  expresses  the  expectation  that  as  the  components  conduct  their 
analyses,  changes  and  improvements  to  the  guidance  will  be  made; 

•  is  careful  to  ensure  that  the  components  understand  base  parameters 
for  conducting  the  analysis  so  it  is  conducted  in  a  manner  that  is 
methodologically  sound; 

•  encourages  consultation  with  experts,  such  as  statisticians  and  human 
resources  researchers,  to  assist  with  determining  the  most  suitable 
analytical  models  to  employ,  the  statistical  tools  to  utilize,  and  the 
standards  to  adopt  in  relation  to  understanding,  measuring,  and 
reporting  significant  findings;  and 

•  makes  responsibility  for  conducting  the  analysis  a  shared 
responsibility  between  various  offices,  including  the  components’ 
legal,  equal  employment  opportunity,  and  human  resources  offices,  but 
notes  that  the  components  should  consider  tasking  their  Office  of 
General  Counsel  or  Office  of  the  Judge  Advocate  General,  whose  staff 
are  well  positioned  to  ensure  that  the  components  are  in  compliance 
with  applicable  statutes,  regulations,  and  policies,  with  primary 
responsibility  for  conducting  the  analysis  and  ensuring  that  adequate 
resources  are  provided  in  support  of  the  function. 

While  issuance  of  the  May  2009  guidance  represents  a  noteworthy  step,  the 
guidance  does  not  address  all  steps  suggested  by  the  Equal  Employment 
Opportunity  Commission  for  identifying  and  addressing  potential  barriers 
to  fair,  consistent,  and  equitable  ratings.  The  Equal  Employment 
Opportunity  Commission’s  Instructions  to  Federal  Agencies  for  EEO 
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Management  Directive  71 533  provides  guidance  that  agencies  can  use  in 
identifying  and  addressing  potential  barriers.34  The  instructions  state  that 
“barrier  identification  and  elimination  is  the  process  by  which  agencies 
uncover,  examine,  and  remove  barriers  to  equal  participation  at  all  levels 
of  the  workforce.”  Management  Directive  715  further  states  that  “where  it 
is  determined  that  an  identified  barrier  serves  no  legitimate  purpose  with 
respect  to  the  operation  of  an  agency,  this  Directive  requires  that  agencies 
take  immediate  steps  to  eliminate  the  barrier.”  In  conducting  their 
analysis,  the  components’  data  may  uncover  barriers  or  other  potential 
problems.  However,  understanding  why  the  barrier  or  problem  exists,  or 
what  to  do  to  address  it,  may  require  that  the  components  take  additional 
steps.  To  identify  and  eliminate  potential  barriers,  the  directive  outlines  a 
four-step  process:  (1)  analyzing  workforce  data  to  identify  potential 
barriers;  (2)  investigating  actual  barriers  and  their  causes;  (3)  eliminating 
barriers,  which  includes  devising  a  plan  for  improvement  and  developing 
overall  objectives  for  barrier  elimination,  with  corresponding  action  items, 
responsible  personnel,  and  target  dates;  and  (4)  assessing  the  success  of 
the  plan.  The  PEO’s  guidance  aims  to  promote  a  degree  of  uniformity  and 
standardization  in  conducting  postdecisional  analyses.  However,  the 
PEO’s  guidance  does  not  specify  what  process  the  components  should 
follow  to  investigate  potential  barriers  and  their  causes,  nor  does  it  specify 
a  process  for  eliminating  barriers  that  are  found.  By  not  specifying  such 
steps  in  its  guidance,  the  components  may  not  follow  a  consistent 
approach  when  investigating  barriers,  which  could  hinder  their  efforts  to 
eliminate  them. 

While  not  predecisional,  we  recognize  that  DOD’s  approach  does  provide 
some  benefits,  some  of  which  are  similar  to  those  of  a  predecisional 
analysis.  For  example,  DOD’s  approach  lays  out  a  method  of  analyzing 
ratings  that  would  address  some  of  the  merit  principles  in  5  U.S.C.  § 

2301 — for  example,  that  employees  should  receive  fair  and  equitable 


33Management  Directive  715  is  the  policy  guidance  which  the  Equal  Employment 
Opportunity  Commission  provides  to  federal  agencies  for  their  use  in  establishing  and 
maintaining  effective  affirmative  programs  of  equal  employment  opportunity  under  Section 
717  of  Title  VII  of  the  Civil  Rights  Act  of  1964,  as  amended,  42  U.S.C.  §  2000(e)  et  seq.,  and 
effective  affirmative  action  programs  under  Section  501  of  the  Rehabilitation  Act  of  1973, 
as  amended  by  Pub.  L.  No.  99-506,  §  103  (1986). 

34Management  Directive  715  defines  a  barrier  as  “an  agency  policy,  principle,  practice  or 
condition  that  limits  or  tends  to  limit  employment  opportunities  for  members  of  a 
particular  gender,  race  or  ethnic  background  or  for  an  individual  (or  individuals)  based  on 
disability  status.” 
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treatment  in  all  aspects  of  personnel  management  and  that  employees 
should  be  protected  against  arbitrary  action,  personal  favoritism,  or 
coercion  for  partisan  political  purposes.36  However,  as  stated  previously, 
because  DOD  does  not  specify  what  process  the  components  should 
follow  to  investigate  and  eliminate  potential  barriers,  the  components  may 
not  follow  a  consistent  approach,  which  could  hinder  their  efforts  to 
ensure  fair,  consistent,  and  equitable  ratings. 

While  DOD  continues  to  take  steps  to  ensure  a  reasonable  amount  of 
transparency  in  its  implementation  of  NSPS,  concerns  about  the  overall 
transparency  of  the  system  continue  to  exist.  To  improve  the  transparency 
of  the  NSPS  performance  management  system,  our  September  2008  report 
recommended  that  DOD  require  commands  to  publish  the  final  rating 
results  to  employees.  DOD  concurred  with  our  recommendation  and,  in 
November  2008,  amended  its  NSPS  regulations  and  implementing 
issuances  to  require  commands  to  publish  the  final  rating  results  to 
employees.  Under  DOD’s  revised  guidance,  performance  review 
authorities  are  required  to  communicate  the  general  pay  pool  results  to  the 
NSPS  workforce  in  writing.  At  a  minimum,  this  includes  the  number  of  pay 
pools  (if  aggregate  pay  pool  results  are  necessary),  the  number  of 
employees  rated,  the  rating  and  share  distributions,  the  average  rating,  the 
average  share  assignment,  the  share  value  (or  average  share  value),  and 
the  average  payout  expressed  as  a  percentage  of  base  salary.  At  the  eight 
locations  we  visited,  we  found  that  each  of  pay  pools  shared  this 
information  with  employees  following  the  2008  NSPS  performance 
management  cycle. 

DOD  continues  to  take  other  steps  to  ensure  a  reasonable  amount  of 
transparency  of  the  NSPS  performance  management  system.  In  May  2009, 
the  PEO  launched  “NSPS  Connect,”  a  centralized  Web  portal  for 
employees,  managers,  and  others  to  access  NSPS  products,  such  as  online 
training  courses,  fact  sheets,  tips  sheets,  and  information  on  the  system’s 
automated  tools.  The  PEO  also  continues  to  take  steps  to  collect  and  share 
lessons  learned  on  the  department’s  experiences  implementing  NSPS  and 
facilitate  lessons  learned  briefings  with  the  components  following  each 
performance  management  cycle.  Further,  the  PEO  continues  to  report 
periodically  on  internal  assessments  and  employee  survey  results  relating 
to  the  NSPS  performance  management  system.  For  example,  in  May  2009, 
the  PEO  published  the  results  of  its  2008  evaluation  of  NSPS.  Similarly,  as 


365  U.S.C.  §  2301(b)(2)  and  (b)(8)(A). 
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Meaningful  Distinctions  in 
Individual  Employee 
Performance 


we  previously  reported,  DOD  posts  the  results  of  its  survey  of  civilian 
employees  on  a  Web  site  that  is  accessible  to  DOD  employees, 
supervisors,  and  managers. 

Officials  at  each  of  the  eight  locations  we  visited  told  us  that  efforts  were 
being  made  to  help  ensure  transparency  of  the  NSPS  performance 
management  system  and  the  rating  process.  According  to  officials,  among 
the  steps  being  taken  to  help  ensure  transparency,  for  example,  were 
establishing  multiple  communities  of  practice,  disseminating  business 
rules  and  other  guidance  on  NSPS  to  employees  and  managers  under  the 
system,  and  publishing  monthly  newsletters  on  NSPS.  At  seven  of  the 
locations,  officials  told  us  that  town  hall  meetings  were  held  to  keep 
employees  informed  of  NSPS-related  developments,  and  officials  at  six 
locations  told  us  that  mock  pay  pool  panels  were  held  to  show  employees 
how  the  pay  pool  process  works. 

However,  our  site  visits  revealed  some  concerns  about  the  overall 
transparency  of  the  system.  For  example,  at  three  locations  officials 
expressed  concerns  over  a  lack  of  transparency  with  regard  to  their  pay 
pools’  business  rules,  indicating  that  their  business  rules  either  had  not 
been  published  or  were  published  well  after  the  performance  management 
cycle  had  begun.  At  one  location,  pay  pool  panel  members  told  us  that 
though  it  was  6  months  into  the  current  performance  management  cycle, 
they  did  not  yet  have  copies  of  their  business  rules.  Similarly,  rating 
officials  at  four  locations  told  us  that  they  did  not  understand  what 
constituted  a  rating  of  “4”  because  neither  their  pay  pools  nor  business 
rules  provided  clear  criteria. 

Although  our  site  visits  revealed  concerns  over  the  extent  to  which 
meaningful  distinctions  in  individual  employee  performance  are  being 
made  under  NSPS,  DOD  has  taken  some  steps  that  include  addressing  a 
recommendation  we  made  in  our  2008  report  aimed  at  encouraging  the  use 
of  all  available  rating  categories.  Specifically,  at  all  eight  locations  we 
visited,  officials  told  us  that  they  did  not  believe  NSPS  was  being 
implemented  in  a  manner  that  encouraged  meaningful  distinctions  in 
individual  employee  performance.  For  example,  officials  at  four  locations 
told  us  that  they  were  hesitant  to  give  ratings  lower  than  a  “3,”  and 
officials  at  six  locations  told  us  that  they  believed  that  there  was  a  forced 
distribution  of  the  ratings  or  manipulation  of  the  ratings  in  order  to 
achieve  a  predetermined  quota.  Further,  at  one  location  we  found  that 
management  stressed  the  importance  of  maintaining  employee  share  value 
at  above  2.0,  which  would  result  in  a  higher  payout,  regardless  of  the 
employee’s  rating.  According  to  the  PEO,  any  effort  to  limit  share  value  is 
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a  roundabout  way  to  establish  preset  limits  on  ratings  and  would 
constitute  forced  distribution,  which  the  NSPS  regulations  prohibit.36 
However,  in  response  to  concerns  about  the  potential  for  the  forced 
distribution  of  performance  ratings  under  NSPS,  in  April  2009  the 
department  posted  to  its  NSPS  Web  site  a  fact  sheet  emphasizing  that  the 
forced  distribution  of  ratings  is  prohibited  under  NSPS  and  describing  how 
meaningful  distinctions  in  performance  are  made  under  the  system.  DOD’s 
fact  sheet  provides  guidance  specifying  what  constitutes  the  forced 
distribution  of  ratings,  why  the  forced  distribution  of  ratings  is  prohibited, 
how  use  of  standard  performance  indicators  minimizes  the  potential  for 
individual  bias  or  favoritism,  and  how  organizations  can  best  apply  this 
information  when  rating  and  rewarding  employee  performance  under 
NSPS.  DOD  has  also  noted  that  if  employees  believe  their  rating  did  not 
result  from  meaningful  distinctions  or  are  unfair,  the  system  affords  them 
the  opportunity  to  challenge  their  ratings  through  a  formal  process  known 
as  reconsideration.  See  appendix  II  for  further  information  on  the  NSPS 
reconsideration  process. 


365  C.F.R.  §  9901.412(a). 
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Although  DOD  Monitors 
Some  Aspects  of  the 
Implementation  of  NSPS, 

It  Does  Not  Monitor  How 
the  Safeguards  Specifically 
Are  Implemented  across 
the  Department 


Within  DOD,  both  the  PEO  and  the  components  are  responsible  for 
monitoring  the  implementation  of  NSPS.  As  part  of  its  efforts  to  monitor 
the  implementation  of  NSPS,  the  PEO  conducts  broad,  annual  evaluations 
of  NSPS  to  determine  whether  the  system  is  on  track  to  achieve  certain 
goals,  or  key  performance  parameters,37  by,  among  other  things, 
monitoring  employee  perceptions  of  the  system  using  DOD’s  survey  of 
civilian  employees.  In  May  2009,  the  PEO  published  its  first  evaluation  of 
NSPS,  which  focused  on  determining  whether  NSPS,  as  implemented  in 
spiral  1  organizations,  was  on  track  to  achieve  some  of  the  goals  specified 
by  DOD  and  if  any  improvements  were  needed.38  While  some  of  DOD’s 
goals  for  the  system  lend  themselves  to  comparisons  to  the  safeguards — 
for  example,  one  of  its  goals  is  ensuring  a  credible  and  trusted  system 
(which  could  align  with  the  transparency  safeguard) — PEO  officials  stated 
that  to  date,  their  focus  has  been  limited  to  understanding  how  the 
components  have  generally  implemented  NSPS  and  not  on  monitoring  or 
assessing  the  components’  implementation  of  the  safeguards. 


With  respect  to  the  components,  DOD’s  implementing  issuances  state  that 
the  heads  of  DOD  components  are  accountable  for  the  manner  in  which 
management  in  their  organizations  carries  out  NSPS  policy,  procedures, 
and  guidance.  However,  we  found  that  only  the  Army  has  taken  steps 
similar  to  the  PEO  to  assess  whether  it  is  on  track  to  achieve  DOD’s  goals 


3 ‘DOD  has  established  certain  goals  for  the  NSPS  performance  management  system,  known 
as  key  performance  parameters.  DOD  defines  a  key  performance  parameter  as  “a  capability 
or  characteristic  that  is  so  significant  that  failure  to  meet  a  minimum  ‘threshold’  can  be 
cause  for  that  element,  concept  or  system  to  be  reevaluated,  or  the  program  to  be 
reassessed  or  terminated.”  DOD  originally  identified  six  key  performance  parameters  for 
NSPS:  (1)  High  Performing  Workforce  and  Management — whether  employees  and 
supervisors  are  compensated  and  retained  based  on  their  performance  and  contribution  to 
mission;  (2)  Agile  and  Responsive  Workforce  and  Management — whether  the  workforce 
can  be  easily  sized,  shaped,  and  deployed  to  meet  changing  mission  requirements; 

(3)  Credible  and  Trusted  System — whether  the  system  assures  openness,  clarity, 
accountability  and  adherence  to  the  public  employment  principles  of  merit  and  fitness; 

(4)  Fiscally  Sound  System — whether  aggregate  increases  in  civilian  payroll,  at  the 
appropriations  level,  will  conform  to  Office  of  Management  and  Budget  fiscal  guidance, 
and  whether  managers  will  have  flexibility  to  manage  to  budget  at  the  unit  level; 

(5)  Supporting  Infrastructure — whether  information  technology  support,  training,  and 
change  management  plans  are  available  and  funded;  and  (6)  Schedule — whether  NSPS  will 
be  operational  and  stable  before  November  2009. 

38DOD’s  2008  evaluation  focused  only  on  spiral  1  organizations  under  NSPS — which 
included  a  total  of  108,758  employees.  Also,  according  to  the  PEO,  externally  driven  delays 
to  converting  employees  to  NSPS  subsequently  resulted  in  the  department’s  elimination  of 
the  sixth  key  performance  parameter,  Schedule.  Therefore,  DOD’s  2008  evaluation  did  not 
assess  the  extent  to  which  the  department  was  on  track  to  achieve  this  particular  goal. 
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for  the  system — the  other  components  have  not  done  so.  Furthermore, 
none  of  the  components  monitors  how  the  safeguards  specifically  are 
implemented  within  their  organizations  because  there  is  no  requirement  to 
do  so. 

We  have  previously  reported  that  transitioning  to  a  more  performance- 
oriented  pay  system  is  a  huge  undertaking  that  requires  constant 
monitoring.39  Further,  in  our  2008  assessment  of  NSPS,  we  noted  that 
continued  monitoring  of  the  safeguards  was  needed  to  ensure  that  DOD’s 
actions  were  effective  as  implementation  of  NSPS  proceeded.  While  DOD’s 
efforts  to  date  have  helped  provide  decision  makers  with  some 
information  on  how  the  department  is  implementing  NSPS,  including 
potential  areas  for  changes  or  improvements,  they  do  not  provide  decision 
makers  in  DOD  and  the  Congress  with  information  to  determine  whether 
the  safeguards  specifically  have  been  implemented  effectively.  Without 
monitoring  the  safeguards’  implementation,  decision  makers  in  DOD  and 
the  Congress  lack  information  that  could  be  used  to  determine  whether 
the  department’s  actions  are  effective  and  whether  the  system  is  being 
implemented  in  a  fair,  equitable,  and  credible  manner.  For  example,  in 
conducting  our  review,  we  identified  some  issues  related  to  the 
implementation  of  the  safeguards  that  illustrate  the  need  to  monitor  their 
implementation.  These  include  the  following: 

•  Ensuring  that  adequate  agency  resources  are  allocated  for  the 
system’s  design,  implementation,  and  administration.  We  found 
that  each  of  the  components  generally  lacks  visibility  over  the  reasons 
why  organizations  have  supplemented  their  pay  pool  funds.  For 
example,  Air  Force  NSPS  program  officials  told  us  that  for  the  2009 
payout,  while  they  knew  that  8  of  the  Air  Force’s  18  major  commands 
supplemented  their  pay  pool  funds,  they  did  not  know  the  specific 
reasons  why.  According  to  PEO  officials,  organizations  might  elect  to 
supplement  their  pay  pool  funds  for  a  variety  of  reasons — for  example, 
to  recruit  or  retain  employees  or  to  compete  with  other  organizations 
for  talent.  However,  because  they  do  not  understand  the  reasons  why 
pay  pools  supplement  their  pay  pool  funding — which  could  help  DOD 
and  the  components  understand  the  extent  to  which  adequate 
resources  have  been  allocated  to  the  system — decision  makers  cannot 
identify  or  assess  any  trends  in  these  practices.  Indeed,  DOD’s  2008 


39GAO,  Human  Capital:  Symposium  on  Designing  and  Managing  Market-Based  and 
More  Performance- Orie?ited  Pay  Systems,  GAO-05-832SP  (Washington,  D.C.:  July  27, 
2005). 
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evaluation  report  notes  that  some  employees  in  organizations  that 
supplemented  their  pay  pools’  funding  questioned  whether  the  higher 
funding  levels  could  be  sustained  over  the  long  term. 

•  Ensuring  reasonable  transparency  of  the  system  and  its 

operation.  We  found  evidence  that  three  of  the  pay  pools  we  visited 
deviated  from  their  business  rules  during  the  last  performance 
management  cycle,  indicating  a  lack  of  transparency  of  the 
performance  management  process  in  some  instances.  DOD’s  guidance 
states  that  a  pay  pool’s  business  rules  are  the  guiding  principles  or 
ground  rules  that  are  used  throughout  the  pay  pool  process,  that  pay 
pool  panels  should  establish  these  principles  and  hold  one  another 
accountable  for  following  them,  and  that  a  pay  pool’s  policies — which 
would  include  its  business  rules — will  be  made  available  to  employees 
before  the  end  of  the  performance  cycle.  However,  at  three  of  the 
locations  we  visited,  we  found  evidence  that  the  pay  pool  had  deviated 
from  its  business  rules  during  the  course  of  the  last  performance 
management  cycle.  For  example,  at  one  location  the  pay  pool’s 
business  rules  required  all  recommended  ratings  be  reviewed,  noting 
that  the  pay  pool  panel  will  ensure  that  all  employees  receive 
appropriate  consideration  and  that  ratings  are  fair  and  consistent. 
However,  officials  we  spoke  with  at  that  location  told  us  that  they  did 
not  review  all  recommended  ratings  in  accordance  with  their  business 
rules.  Rather,  only  the  recommended  ratings  of  “4”  or  “5”  were 
reviewed.  As  another  example,  we  found  evidence  that  a  pay  pool  at 
another  location  used  different  criteria  than  what  was  specified  in  its 
business  rules  for  allocating  the  number  of  shares  to  employees. 
According  to  component-level  NSPS  program  officials,  in  order  to 
ensure  transparency  of  the  system,  pay  pools  should  not  deviate  from 
their  business  rules  once  those  rules  are  published.  However,  none  of 
the  components  requires  its  pay  pools  to  notify  it  when  such  an  event 
occurs,  or  of  the  reasons  why,  though  doing  so  could  help  provide 
decision  makers  with  information  on  the  extent  to  which  pay  pools  are 
implementing  the  system  in  a  manner  that  is  transparent  to  employees. 
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Although  DOD 
Civilian  Employees 
under  NSPS  Have 
Mixed  Views  about 
the  System,  DOD  Has 
Not  Yet  Developed 
and  Implemented  a 
Plan  to  Address 
Employees’  Negative 
Perceptions  of  Some 
Aspects  of  the  System 


DOD  civilian  personnel  have  mixed  perceptions  about  NSPS,  and  although 
DOD  has  taken  some  steps  toward  addressing  employees’  concerns,  it  has 
not  yet  developed  and  implemented  an  action  plan  to  address  areas  where 
employees  express  negative  perceptions  of  the  system.  DOD’s  most  recent 
survey  of  civilian  employees  reveals  that  NSPS  employees  have  mixed 
perceptions  about  NSPS.  The  responses  to  questions  specifically  asking 
about  NSPS  show  positive  perceptions  about  some  aspects  of 
performance  management  under  NSPS,  including  connecting  pay  to 
performance,  but  show  negative  perceptions  about  other  aspects  of 
performance  management,  such  as  the  appraisal  process.  Further,  the 
most  recent  data  indicate  that  the  perceptions  of  those  employees  who 
have  worked  under  NSPS  the  longest  appear  to  have  remained  largely 
unchanged  from  the  negative  perceptions  we  reported  in  2008.  Moreover, 
during  discussion  groups  we  held  at  eight  locations  outside  the  continental 
United  States,  civilian  employees  and  supervisors  expressed  consistent 
concerns  and  negative  perceptions  of  NSPS  that  are  similar  to  those 
identified  in  our  2008  report,  although  they  also  identified  positive  aspects 
of  the  system.  DOD  has  taken  some  steps  to  address  employees’  negative 
perceptions  of  the  system;  however,  the  department  has  yet  to  develop 
and  implement  an  action  plan  that  meets  the  intent  of  our  prior 
recommendation  because  it  does  not  specify  such  things  as  the  actions 
DOD  intends  to  take,  who  will  be  responsible  for  taking  the  action,  and  the 
timelines  for  doing  so.  We  continue  to  believe  that  implementing  such  an 
action  plan  is  important  and  note  that  doing  so  would  be  a  step  that  DOD 
could  take  to  involve  employees  in  the  system’s  implementation — which  is 
one  of  the  safeguards  we  previously  discussed. 


NSPS  Employees  Express 
Mixed  Perceptions  about 
the  System 


NSPS  Employees  in  All  Spirals 
Have  Positive  Views  of  Some 
Aspects  of  Performance 
Management  but  Negative 
Views  of  Other  Aspects 


According  to  DOD’s  most  recent  survey  data,  some  NSPS  employees 
recognize  that  positive  aspects  of  performance  management,  such  as 
connecting  pay  to  performance,  exist  under  the  system.  For  example,  as 
shown  in  table  2,  DOD’s  survey  data  for  2008  indicate  that  an  estimated  38 
percent  of  NSPS  employees  agree  that  differences  in  their  performance  are 
recognized  in  meaningful  ways,  as  compared  with  an  estimated  33  percent 
of  non-NSPS  employees.  Further,  an  estimated  42  percent  of  NSPS 
employees  agree  that  pay  raises  depend  on  how  well  employees  perform 
their  jobs,  as  compared  with  an  estimated  25  percent  of  non-NSPS 
employees.  When  asked  about  how  poor  performers  are  dealt  with,  an 
estimated  34  percent  of  NSPS  employees,  versus  an  estimated  27  percent 
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of  non-NSPS  employees,  agreed  that  steps  are  taken  to  deal  with  poor 
performers. 


Table  2:  Estimated  Percentage  of  Responses  from  NSPS  and  Non-NSPS  Employees  to  Selected  Questions  about 
Performance  Management  in  the  2008  Status  of  Forces  Survey  of  Civilian  Employees 


Employee 

Percentage  responding 

Performance  management  questions 

description 

“Agree” 

“Neither/nor” 

“Disagree” 

Differences  in  performance  are  recognized  in  meaningful 

NSPS 

38 

31 

28 

ways. 

Non-NSPS 

33 

32 

31 

Pay  raises  depend  on  how  well  employees  perform  their 
jobs. 

NSPS 

42 

26 

30 

Non-NSPS 

25 

29 

42 

Performance  standards/expectations  are  directly  related  to 
the  organization's  mission. 

NSPS 

70 

21 

9 

Non-NSPS 

65 

25 

10 

In  my  work  unit,  steps  are  taken  to  deal  with  a  poor 
performer  who  cannot  or  will  not  improve. 

NSPS 

34 

31 

30 

Non-NSPS 

27 

30 

38 

My  bonus  and  cash  awards  depend  on  how  well  1  perform 
my  job. 

NSPS 

61 

17 

23 

Non-NSPS 

55 

20 

25 

Source:  Defense  Manpower  Data  Center. 

Note:  The  estimated  percentages  are  based  on  a  95  percent  confidence  interval  and  the  margin  of 
error  is  within  +/- 1  percent.  The  response  categories  are  collapsed  for  positive  ("agree")  and  negative 
("disagree")  responses.  That  is,  "agree"  is  the  estimated  percentage  of  employees  who  responded 
either  "agree"  or  "strongly  agree,"  while  "disagree"  is  the  estimated  percentage  of  employees  who 
responded  either  "disagree"  or  "strongly  disagree. "Totals  may  sum  to  less  than  or  more  than  100 
percent  as  a  result  of  collapsing  the  positive  and  negative  response  categories. 


In  comparison,  an  estimated  47  percent  of  non-NSPS  employees,  as 
compared  with  an  estimated  44  percent  of  NSPS  employees,  agreed  that 
their  current  performance  appraisal  system  motivates  them  to  perform 
well.  Further,  an  estimated  34  percent  of  non-NSPS  employees,  as 
compared  with  an  estimated  29  percent  of  NSPS  employees,  agreed  that 
their  performance  appraisal  system  improves  organizational  performance. 
Table  3  shows  additional  comparisons  between  NSPS  and  non-NSPS 
employee  responses  to  questions  about  performance  appraisals. 


Page  36 


GAO-10-102  Human  Capital 


Table  3:  Estimated  Percentage  of  Responses  from  NSPS  and  Non-NSPS  Employees  to  Selected  Questions  about 
Performance  Appraisals  in  the  2008  Status  of  Forces  Survey  of  Civilian  Employees 


Performance  appraisal  questions 

Employee 

description 

Percentage  responding 
“Agree”  “Neither/nor” 

“Disagree” 

Performance  appraisal  is  a  fair  reflection  of  performance. 

NSPS 

62 

18 

19 

Non-NSPS 

65 

19 

14 

In  my  most  recent  appraisal,  1  understood  what  1  had  to  do 

NSPS 

59 

16 

22 

Lo  be  rated  at  different  performance  levels. 

Non-NSPS 

65 

17 

14 

My  current  performance  appraisal  system  motivates  me  to 

NSPS 

44 

28 

28 

perform  well. 

Non-NSPS 

47 

31 

22 

The  performance  appraisal  system  1  am  under  improves 

NSPS 

29 

36 

36 

organizational  performance. 

Non-NSPS 

34 

40 

26 

Procedures  for  reconsidering  performance  appraisal 

NSPS 

37 

45 

19 

ratings  are  fair. 

Non-NSPS 

38 

45 

17 

Source:  Defense  Manpower  Data  Center. 


Note:  The  estimated  percentages  are  based  on  a  95  percent  confidence  interval  and  the  margin  of 
error  is  within  +/- 1  percent.  The  response  categories  are  collapsed  for  positive  ("agree")  and  negative 
("disagree")  responses.  That  is,  "agree”  is  the  estimated  percentage  of  employees  who  responded 
either  "agree"  or  "strongly  agree,"  while  "disagree”  is  the  estimated  percentage  of  employees  who 
responded  either  "disagree"  or  "strongly  disagree. "Totals  may  sum  to  less  than  or  more  than  100 
percent  as  a  result  of  collapsing  the  positive  and  negative  response  categories. 

Perceptions  of  Employees  In  our  first  assessment  of  NSPS,  we  reported  that  the  results  of  DOD’s 

under  NSPS  the  Longest  Status  of  Forces  Survey  of  Civilian  Employees  indicated  that  the 

Remain  Largely  Unchanged  perceptions  of  employees  who  had  been  under  the  system  the  longest  had 

since  GAO’s  2008  Report  become  more  negative  on  questions  related  to  performance  management. 

However,  the  results  of  DOD’s  most  recent  administration  of  the  survey  in 
2008  indicate  that  spiral  1.1  employee  perceptions  are  about  the  same  as 
the  May  2007  survey,  as  shown  in  table  4.  For  example,  from  the 
November  2006  through  February  2008  administrations  of  DOD’s  survey, 
the  percentage  of  spiral  1.1  employees  that  agreed  that  they  understood 
what  they  had  to  do  to  be  rated  at  a  different  performance  level  declined 
from  an  estimated  59  percent  in  November  2006  to  an  estimated  53  percent 
in  May  2007,  then  remained  largely  unchanged  in  February  2008  at  an 
estimated  54  percent.40 


40The  first  Web-based  Status  of  Forces  Survey  was  conducted  in  October  2003.  Regular 
administrations  of  the  Status  of  Forces  Survey  of  Civilian  Employees  occurred  every  6 
months  from  October  2004  through  November  2006,  and  annual  administrations 
commenced  in  2007. 
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Table  4:  Estimated  Percentage  of  Responses  from  Spiral  1.1  Employees  to  Selected  Questions  about  Performance 
Management  from  the  May  2006,  November  2006,  May  2007,  and  February  2008  Administrations  of  the  Status  of  Forces 

Survey  of  Civilian  Employees 

Performance 

Survey 

Percentage  of  spiral  1.1  employees  responding 

management  questions 

administration 

“Agree” 

“Neither/  nor”  “Disagree” 

Performance  appraisal  is  a  fair  reflection 
of  performance. 

May  2006 

67 

20 

12 

November  2006 

59 

22 

16 

May  2007 

52 

21 

25 

February  2008 

54 

20 

25 

Differences  in  performance  are 
recognized  in  meaningful  ways. 

May  2006 

37 

33 

28 

November  2006 

35 

31 

31 

May  2007 

32 

31 

34 

February  2008 

35 

30 

32 

In  most  recent  appraisal,  1  understood 
what  1  had  to  do  to  be  rated  at  different 
performance  levels. 

May  2006“ 

n/a 

n/a 

n/a 

November  2006 

59 

17 

22 

May  2007 

53 

17 

29 

February  2008 

54 

17 

27 

Source:  Defense  Manpower  Data  Center. 


Note:  The  estimated  percentages  are  based  on  a  95  percent  confidence  interval  and  the  margin  of 
error  is  within  +/-  2  percent.  The  response  categories  are  collapsed  for  positive  ("agree")  and  negative 
("disagree")  responses.  That  is,  "agree"  is  the  estimated  percentage  of  employees  who  responded 
either  "agree"  or  "strongly  agree,"  while  "disagree"  is  the  estimated  percentage  of  employees  who 
responded  either  "disagree"  or  "strongly  disagree. "Totals  may  sum  to  less  than  or  more  than  100 
percent  as  a  result  of  collapsing  the  positive  and  negative  response  categories. 

’This  question  was  notasked  on  the  May  2006  administration  of  the  Status  of  Forces  Survey  of 
Civilian  Employees. 


In  addition,  when  asked  about  the  overall  impact  that  NSPS  will  have  on 
personnel  practices  in  DOD,  spiral  1.1  employees’  perceptions  have 
become  significantly  more  negative  since  first  converting  to  NSPS  in  2006, 
but  showed  little  change  between  the  May  2007  and  February  2008 
surveys.  Specifically,  the  results  of  the  2008  survey  indicate  that  that  an 
estimated  22  percent  of  spiral  1.1  employees  responded  that  the  overall 
impact  of  NSPS  on  the  department’s  personnel  practices  would  be 
positive,  as  compared  to  an  estimated  23  percent  in  May  2007  and  an 
estimated  25  percent  in  November  2006.  Table  5  shows  a  comparison  of 
spiral  1.1  employee  responses  over  time  about  the  overall  impact  of  NSPS 
on  personnel  practices  in  DOD. 
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Table  5:  Estimated  Percentage  of  Responses  from  Spiral  1.1  Employees  about  the  Overall  Impact  of  the  National  Security 
Personnel  System  from  the  May  2006,  November  2006,  May  2007,  and  February  2008  Administrations  the  Status  of  Forces 
Survey  of  Civilian  Employees 


Survey  _ 

Percentage  of  spiral  1.1  employees  responding 

Overall  Impact  question 

administration 

“Positive” 

“Neither/  nor” 

“Negative” 

Overall,  whattype  of  impact  will  NSPS 
have  on  personnel  practices  in  the 
DOD? 

May  2006 

40 

35 

25 

November  2006 

25 

32 

42 

May  2007 

23 

30 

48 

February  2008 

22 

28 

50 

Source:  Defense  Manpower  Data  Center. 


Note:  The  estimated  percentages  are  based  on  a  95  percent  confidence  interval  and  the  margin  of 
error  is  within  +/-  3  percent.  The  response  categories  are  collapsed  for  positive  and  negative 
responses.  That  is,  "positive"  is  the  estimated  percentage  of  employees  who  responded  either 
"positive"  or  "very  positive,"  while  "negative"  is  the  estimated  percentage  of  employees  who 
responded  either  "negative"  or  "very  negative. "Totals  may  sum  to  less  than  or  more  than  100  as  a 
result  of  collapsing  the  positive  and  negative  response  categories. 


NSPS  Employees  and 
Supervisors  in  Our 
Discussion  Groups 
Expressed  Consistent 
Concerns 


As  with  our  first  review  of  NSPS,  DOD  civilians  in  our  discussion  groups  at 
locations  outside  the  continental  United  States  continue  to  express  wide- 
ranging  but  consistent  concerns  about  the  NSPS  performance 
management  system.  Although  the  results  of  our  discussion  groups  are  not 
generalizable  to  the  entire  population  of  DOD  civilian  employees,  the 
themes  that  emerged  provide  valuable  insight  into  employees’  perceptions 
about  the  implementation  of  NSPS  thus  far. 

Specifically,  during  these  discussion  groups,  participants  at  six  locations 
told  us  that  they  were  initially  optimistic  about  the  intent  of  NSPS  and  the 
concept  of  pay  for  performance.  Further,  some  participants  indicated  that 
they  remain  positive  about  the  amount  of  performance  pay  and  flexibilities 
afforded  to  supervisors  to  rate  their  employees  under  the  system. 

However,  as  with  our  first  review,  discussion  group  participants  at  all 
eight  locations  we  visited  primarily  expressed  frustration  with  and 
concern  about  certain  aspects  of  NSPS  implementation  and  the  system. 
The  prevalent  themes  that  emerged  during  our  discussion  groups  include 
concerns  over  (1)  the  negative  impact  of  NSPS  on  employee  motivation 
and  morale,  (2)  the  excessive  amount  of  time  spent  navigating  the 
performance  management  process,  (3)  challenges  with  job  objectives, 

(4)  factors  undermining  employee  confidence  in  the  system,  and 

(5)  factors  unrelated  to  job  performance  affecting  employees’  final 
performance  ratings.  As  we  noted  in  2008,  the  themes  that  emerged  during 
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Concerns  about  a  Negative 
Impact  on  Motivation  and 
Morale 


our  discussion  group  sessions  are  not  surprising.  Our  prior  work,  as  well 
as  that  of  OPM,41  has  recognized  that  organizational  transformations,  such 
as  the  adoption  of  a  new  performance  management  system,  often  entail 
fundamental  and  radical  changes  that  require  an  adjustment  period  to  gain 
employees’  trust  and  acceptance.  As  a  result,  we  expect  major  change 
management  initiatives  in  large-scale  organizations  to  take  several  years  to 
be  fully  successful. 

A  prevalent  theme  from  our  discussions  with  both  employees  and 
supervisors  was  that  several  aspects  of  NSPS  have  had  a  negative  impact 
on  employee  motivation  and  morale — consistent  with  our  first  assessment 
of  NSPS.  Specifically,  discussion  group  participants  at  all  eight  locations 
discussed  how  various  aspects  of  NSPS — for  example,  their  perception 
that  a  rating  of  “3”  is  only  average — have  negatively  affected  their 
motivation  and  morale.  Discussion  group  participants  at  six  of  the  eight 
locations  also  told  us  they  have  negative  perceptions  of  what  a  rating  of 
“3”  means.  At  five  of  those  locations,  discussion  group  participants  told  us 
that  they  continue  to  believe  that  a  rating  of  “3”  means  only  “average,”  in 
contrast  to  “valued  performer,”  as  it  was  initially  defined  to  the  workforce 
by  DOD. 

Discussion  group  participants  at  five  locations  also  discussed  how 
achieving  a  rating  higher  than  a  “3”  seemed  to  be  an  unattainable  goal.  For 
example,  employees  at  four  locations  told  us  that  they  felt  NSPS  either 
does  not  provide  incentives  for  high  performance  or  encourages  only 
mediocre  performance  from  employees  under  the  system  because  of  the 
high  number  of  employees  receiving  “3”-level  ratings  each  year.  As  another 
example,  supervisors  at  one  location  noted  that  across  the  installation 
there  is  a  general  feeling  that  everyone  receives  a  rating  of  “3,”  and 
therefore  such  a  rating  is  considered  average,  no  matter  how  DOD  defines 
it.  Similarly,  discussion  group  participants  at  seven  locations  told  us  that 
they  felt  it  was  difficult  for  employees  in  certain  positions  to  receive  a 
rating  of  “5”  because  of  the  nature  of  their  work  or  the  perceived  value 
their  management  placed  on  those  positions.  At  one  of  those  locations, 
supervisors  told  us  that  they  felt  such  things  as  how  the  pay  pool’s 
business  rules  were  structured  affected  whether  an  employee  could 
receive  a  high  rating.  At  that  location,  the  pay  pool’s  business  rules 
specified  that  an  employee  must  receive  a  higher-level  award,  such  as  a 


41Office  of  Personnel  Management,  Working  for  America:  Alternative  Personnel  Systems 
in  Practice  and  a  Guide  to  the  Future  (Washington,  D.C.,  October  2005). 
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command  or  agencywide  award,  to  receive  a  rating  of  “5.”  However, 
discussion  group  participants  told  us  that  they  felt  some  employees  were 
not  in  a  position  to  receive  such  awards  because  of  their  positions  or  the 
type  of  work  they  did. 

In  addition,  discussion  group  participants  at  all  eight  locations  we  visited 
expressed  frustration  over  the  amount  of  the  annual  performance  payout 
provided  under  NSPS.  For  example,  discussion  group  participants  noted 
that  they  felt  the  payout  was  not  significant  enough  to  encourage  anything 
other  than  average  performance.  Discussion  group  participants  at  six  of 
the  eight  locations  also  told  us  that  they  felt  NSPS  discourages  employees 
from  seeking  additional  responsibilities  and  opportunities  that  fall  outside 
the  scope  of  their  objectives  because  their  payout  may  not  reflect  their 
additional  work.  In  addition,  discussion  group  participants  at  six  locations 
told  us  that  because  supervisory  positions  under  NSPS  require  such  a 
significant  increase  in  responsibility  and  effort,  and  because  the  maximum 
allowable  pay  increase  for  reassignments  is  capped  at  5  percent,42  some 
employees  may  not  seek  promotion  opportunities.  Similarly,  a  discussion 
group  participant  at  another  location  expressed  frustration  that  some 
employees  only  received  their  payout  in  the  form  of  a  bonus  and  not  an 
increase  in  salary.  Discussion  group  participants  at  three  of  the  eight 
locations  also  expressed  concerns  that  they  felt  performance  payouts 
under  NSPS  tended  to  benefit  higher-paid  employees  at  the  expense  of 
lower-paid  employees.  For  example,  employees  at  one  location  expressed 
concerns  that  in  their  pay  pool,  the  higher  payouts  under  NSPS  seemed  to 
go  to  employees  at  the  top  of  the  pay  bands. 

Concerns  about  an  Excessive 
Amount  of  Time  Spent 
Navigating  the  NSPS 
Performance  Management 
Process 


Another  prevalent  theme  at  seven  of  the  eight  locations  and  also 
highlighted  in  our  first  assessment  was  that  employees  spend  an  excessive 
amount  of  time  navigating  the  performance  management  process.  While 
the  discussion  group  participants’  complaints  about  the  time-  and  labor- 
intensive  nature  of  the  system  were  not  limited  to  any  one  part  of  the 
process,  discussion  group  participants  at  seven  locations  pointed  out  that 
the  time  and  effort  required  to  complete  the  steps  of  the  NSPS 
performance  management  process  were  significantly  greater  than  what 
was  required  of  them  under  previous  systems.  For  example,  one 


42The  NSPS  regulations  (5  C.F.R  §  9901.353)  state  that  an  employee  may  only  receive  up  to 
a  total  of  a  5  percent  cumulative  increase  to  his  or  her  base  salary  in  any  12-month  period 
as  the  result  of  an  employee-initiated  action,  unless  an  exception  is  approved  by  an 
authorized  management  official.  There  are  no  limits  to  the  number  of  times  an  employee 
may  be  reassigned  by  management,  however. 
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Concerns  about  Challenges 
with  Job  Objectives  under 
NSPS 


supervisor  we  spoke  with  speculated  that  his  supervisory  duties  under 
NSPS  took  him  six  times  as  long  to  perform  as  they  had  under  the  GS 
system,  while  another  supervisor  told  us  that  he  may  have  spent  from  45  to 
50  hours  assessing  the  performance  of  three  employees,  a  task  he  could 
have  completed  in  10  hours  under  the  GS  system. 

At  five  of  the  locations  we  visited,  employees  expressed  concerns  about 
NSPS  potentially  affecting  their  ability  to  complete  their  jobs  or  affecting 
the  mission  because  of  what  they  perceived  as  an  excessive  amount  of 
time  required  of  employees  and  supervisors  in  navigating  the  NSPS 
performance  management  process.  In  some  instances,  employees  spoke  of 
impacts  NSPS  was  having  on  their  supervisors,  while  others  spoke  of  their 
own  experiences  navigating  the  NSPS  performance  management  process. 
At  three  locations,  discussion  group  participants  described  how  what  they 
perceived  as  an  excessive  amount  of  time  navigating  the  NSPS 
performance  management  process  affected  their  ability  to  complete  their 
job-related  duties,  requiring  completion  of  some  NSPS  tasks,  such  as  self- 
assessments  and  employee  ratings,  after  work  hours  or  on  weekends.  One 
employee  described  feeling  inundated  with  information  on  NSPS  and  that 
it  was  difficult  to  stay  on  top  of  things  while  simultaneously  performing  his 
job,  while  another  employee  estimated  that  she  spent  about  2  hours  per 
week  on  NSPS-related  tasks.  In  some  instances,  discussion  group 
participants  told  us  that  they  saw  the  potential  for  the  excessive  time 
commitment  required  by  NSPS  to  affect  the  missions  of  their 
organizations.  According  to  one  supervisor,  any  task  that  takes  employees 
away  from  their  daily  work  affects  the  mission,  and  any  task  that  takes  the 
time  and  patience  of  the  command’s  leadership  detracts  from  the  mission. 
Further,  in  discussing  during  a  site  visit  the  potential  for  NSPS  to  impact 
the  organization’s  mission,  one  general  officer  we  spoke  with  described 
NSPS  as  “mission  ineffective.” 

Another  prevalent  theme  that  emerged  from  our  discussions  with  both 
supervisors  and  employees  at  all  eight  locations  was  that  there  are 
challenges  with  employee  job  objectives  under  NSPS.  According  to  DOD, 
the  NSPS  performance  management  system  is  designed  to  provide  a  fair 
and  equitable  method  for  appraising  and  evaluating  performance.  As  part 
of  the  system,  DOD  established  the  concept  of  “job  objectives,”  which  are 
the  required  tasks  of  a  given  job  as  determined  by  managers  and 
supervisors,  and  directed  that  job  objectives  be  developed  and  used  as  the 
standards  for  evaluating  employee  performance.  However,  supervisors 
and  employees  at  each  of  the  eight  locations  discussed  challenges  they 
experienced  developing  their  job  objectives  under  NSPS.  Specifically: 
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•  Although  DOD  guidance  encourages  employees  to  develop  job 
objectives  that  are  specific,  measurable,  aligned,  realistic,  and  timed — 
an  approach  summarized  by  the  acronym  S.M.A.R.T. — employees  and 
supervisors  we  met  at  six  of  eight  locations  discussed  how  they  found 
it  challenging  to  develop  job  objectives  that  are  measurable  or  that 
follow  the  S.M.A.R.T.  approach.  Supervisors  at  one  location  objected 
to  the  S.M.A.R.T.  approach,  particularly  the  “specific”  portion,  because 
they  felt  that  job  objectives  needed  to  be  broad  enough  to  allow 
employees  to  discuss  any  accomplishments  they  make  if  they  complete 
additional  job  activities  or  other  tasks  that  might  arise  during  the  year. 
Supervisors  at  two  locations  discussed  how  the  work  they  did  was 
nebulous  and  unpredictable,  which  made  it  challenging  to  develop  job 
objectives  that  not  only  reflected  the  nature  of  their  work  but  that  they 
could  exceed.  Similarly,  supervisors  at  another  location  expressed 
concerns  that  employees’ job  objectives  may  not  reflect  the  work  they 
do  by  the  end  of  the  performance  management  cycle  because  of 
constant  changes  within  their  organization. 

•  According  to  discussion  group  participants  at  four  locations,  guidance 
for  developing  job  objectives  is  either  limited  or  nonexistent,  which 
may  result  in  different  approaches  to  developing  job  objectives  across 
an  organization.  At  one  of  these  locations,  employees  told  us  that  their 
management  had  not  established  consistent  ground  rules  for 
developing  job  objectives  and  that  as  a  result  some  employees’  job 
objectives  were  based  on  out-of-date  position  descriptions.  One 
organization  we  visited  used  a  mixture  of  mandatory  and  employee- 
specific  job  objectives;  but,  according  to  one  employee,  little  guidance 
exists  to  help  employees  and  supervisors  when  they  need  to  develop 
personalized  job  objectives.  Employees  at  another  location  told  us  that 
there  were  significant  differences  in  the  amount  of  involvement  they 
had  in  developing  their  job  objectives.  For  example,  one  individual  told 
us  that  employees  in  her  office  develop  their  own  objectives,  while 
another  said  employees  in  her  office  are  assigned  mandatory 
objectives  and  were  thus  unable  to  provide  input  into  their  objectives. 

•  Discussion  group  participants  at  six  locations  expressed  concerns  that 
it  can  be  difficult  to  achieve  a  high  rating  for  some  job  objectives. 

Some  locations  we  visited  used  mandatoiy  job  objectives,  which  left 
employees  concerned  that  their  job  objectives  did  not  accurately 
capture  the  full  responsibilities  of  the  work  they  performed.  For 
example,  at  one  location,  a  uniform,  mandatory  supervisory  objective 
accounted  for  half  of  supervisors’  ratings,  which,  according  to  one 
supervisor,  diminished  the  value  of  the  other  responsibilities  they  had. 
The  supervisor  expressed  further  concern  that  some  mandatory  job 
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Concerns  about  Factors  That 
Undermine  Employee 
Confidence  in  the  System 


objectives,  such  as  those  assigned  to  government  purchase  card 
holders,  require  a  pass-fail  evaluation,  making  it  difficult,  if  not 
impossible,  for  the  employee  to  receive  a  high  rating.  In  one  instance,  a 
location  we  visited  required  all  employees  to  be  rated  against  a 
mandatory  safety  objective.  However,  according  to  some  supervisors, 
it  did  not  make  sense  for  everyone  to  have  the  mandatory  safety 
objective  because  for  many  employees,  safety  issues  were  out  of  their 
control. 


During  our  discussion  groups,  participants  at  all  eight  locations  also 
discussed  how  various  factors  undermine  employees’  confidence  in  the 
system  and  its  implementation  thus  far.  For  example,  discussion  group 
participants  at  six  locations  commented  that  they  do  not  believe  that  the 
NSPS  performance  management  system  has  the  ability  to  rate  employees 
fairly.  At  the  locations  we  visited,  discussion  of  these  concerns  centered 
on  such  things  as  the  perception  of  subjectivity  and  the  potential  for 
favoritism  under  NSPS;  a  lack  of  transparency  surrounding  the  pay  pool 
panel  process,  including  a  lack  of  understanding  of  what  employees 
needed  to  do  to  receive  higher  ratings;  and  the  perception  of 
inconsistencies  in  interpretations  of  the  standards  used  to  determine 
employee  ratings. 

One  prevalent  theme  at  all  eight  locations  involved  perceptions  of 
subjectivity,  such  as  the  potential  for  favoritism  under  NSPS  during  the 
rating  and  pay  pool  panel  processes.  At  five  locations,  participants 
discussed  their  frustration  with  how  NSPS  takes  the  responsibility  for 
rating  employees  out  of  the  hands  of  supervisors  and  places  it  in  the  hands 
of  the  pay  pool  panel  members,  who  may  or  may  not  have  any  direct 
knowledge  of  individual  employees’  performance.  One  supervisor  told  us 
that  NSPS  may  inadvertently  favor  employees  who  work  closely  or  are  in 
direct  contact  with  members  of  the  pay  pool  panel  because  those 
individuals  have  direct  knowledge  of  the  employees  and,  sometimes,  their 
performance.  Similarly,  supervisors  at  another  location  told  us  that  they 
did  not  feel  that  their  pay  pool  panel  understood  their  jobs  and  what  they 
do  and  expressed  frustration  that  the  pay  pool  panel  did  not  seem  to  be 
reaching  out  to  their  supervisors  and  higher-level  reviewers  for  additional 
input  on  their  performance.  At  five  of  the  eight  locations,  discussion  group 
participants  also  told  us  that  they  saw  the  potential  for  the  employee- 
supervisor  relationship  to  affect  an  employee’s  rating — either  to  the 
benefit  or  detriment  of  the  employee. 

Another  prevalent  theme  at  six  of  the  eight  locations — a  theme  also 
highlighted  in  our  first  assessment  of  NSPS — was  a  lack  of  transparency 
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Concerns  about  Factors 
Unrelated  to  Performance 
Affecting  Final  Ratings 


and  understanding  of  the  pay  pool  panel  process.  Specifically,  supervisors 
at  two  locations  commented  that  their  organizations’  pay  pool  panel 
processes  were  not  transparent.  A  supervisor  at  one  location  commented 
that  everything  “goes  into  a  black  vacuum,”  while  another  likened  the 
process  to  a  “black  box.”  Employees  at  that  same  location  referred  to  the 
organization’s  pay  pool  panel  process  as  a  “star  chamber,”  where  decisions 
are  made  but  are  not  explained  to  employees.  Employees  and  supervisors 
at  five  locations  expressed  concerns  about  the  amount  of  information  they 
received  from  their  pay  pools  and  about  the  process  itself;  some  desired 
further  information  to  help  them  better  understand  the  pay  pool  panel 
process.  In  addition,  at  six  of  the  eight  locations,  discussion  group 
participants  told  us  that  they  did  not  understand  what  they  needed  to  do  to 
receive  a  higher  rating.  For  example,  an  employee  at  one  location  told  us 
that  she  was  told  by  her  supervisor  that  all  employees  had  to  receive  a 
rating  of  “3”  because  they  would  have  had  to  “walk  on  water”  to  receive  a 
higher  rating.  Discussion  group  participants  at  two  other  locations  also 
discussed  how  “walking  on  water”  was  a  perceived  standard  for  receiving 
a  high  rating  under  NSPS.  At  three  locations,  supervisors  commented  that 
they  were  unclear  about  what  they  could  do  to  help  their  employees 
receive  better  ratings,  while  employees  at  four  locations  were  unclear 
about  what  they  could  do  to  achieve  higher  ratings. 

Discussion  group  participants  at  six  locations  also  raised  concerns  about 
inconsistent  interpretation  of  the  standards  used  when  evaluating  civilian 
employees  under  NSPS.  Discussion  group  participants  reported  concerns 
that  military  supervisors  may  rate  employees  using  more  stringent 
standards  than  their  civilian  counterparts.  Discussion  group  participants 
also  reported  concerns  that  some  military  supervisors  may  not  value  the 
NSPS  performance  management  process  and  sometimes  devote  less  time 
and  effort  to  the  process,  which  could  affect  employees’  ratings.  One 
civilian  supervisor  told  us  that  some  military  supervisors  with  whom  he 
attended  NSPS  training  had  a  much  harsher  perspective  of  employee 
performance  than  their  civilian  counterparts.  For  example,  he  noted  that 
the  military  supervisors  indicated  that  giving  a  rating  of  “1”  or  “2”  was 
acceptable,  whereas  he  believed  civilian  supervisors  would  be  more 
inclined  to  give  an  employee  a  rating  of  “3.”  Employees  also  told  us  that 
they  do  not  believe  some  military  supervisors  value  the  work  of  employees 
who  perform  certain  job  functions,  such  as  providing  child  care  on  an 
installation. 

A  prevalent  theme  expressed  by  discussion  group  participants  at  all  eight 
locations  we  visited  is  that  factors  unrelated  to  performance  may  affect 
employees’  final  performance  ratings.  Such  factors  include  the  existence 
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of  a  forced  distribution  or  quota  of  ratings,  the  writing  ability  of  employees 

and  supervisors,  and  pay  pool  panel  members’  knowledge  of  employees. 

For  example: 

•  Discussion  group  participants  at  all  eight  locations  expressed  concerns 
that  their  pay  pool  panels  used  a  forced  distribution  or  quota  for 
ratings,  which  dictated  the  number  of  ratings  in  each  category  that 
could  be  awarded.43  Employees  at  one  location  told  us  that  they  were 
aware  of  their  management’s  attempts  to  artificially  preserve  a  higher 
share  value  for  employees  by  primarily  awarding  ratings  of  “3,” 
regardless  of  the  employees’  performance.  Further,  at  three  locations 
discussion  group  participants  told  us  that  their  management  told  them 
that  all  employees  should  expect  to  receive  a  rating  of  “3.”  Moreover, 
some  discussion  group  participants  told  us  that  they  doubted  that  their 
actual  performance  had  the  bearing  it  was  supposed  to  have  on  their 
final  ratings,  while  others  felt  the  use  of  a  forced  distribution  or  quotas 
was  in  direct  conflict  with  the  principles  of  pay  for  performance  under 
NSPS.  While  no  discussion  group  participants  we  met  with  were  aware 
of  any  explicit  guidance  provided  to  pay  pool  panels  or  supervisors 
that  limited  the  number  of  certain  ratings  they  assigned  employees, 
employees  and  supervisors  from  at  least  three  locations  believed  that 
informal  guidelines  existed  or  that  pay  pool  panels  or  supervisors  were 
encouraged  to  limit  the  number  of  certain  ratings  they  could  assign. 

•  Discussion  group  participants  at  all  eight  locations  also  expressed 
concerns  that  the  writing  ability  of  employees  and  supervisors  may 
affect  ratings — a  theme  also  highlighted  in  our  first  assessment  of 
NSPS.  Supervisors  at  one  location  likened  the  process  of  developing 
employees’  assessments  under  NSPS  to  a  writing  contest.  Moreover, 
supervisors  told  us  that  they  felt  their  writing  ability  could 
unintentionally  affect  their  employees’  ratings,  noting,  for  example, 
that  a  supervisor’s  ability  to  articulate  an  employee’s  achievements  in 
writing  plays  a  significant  role  in  supporting  a  higher  rating  for  that 
employee.  Employees  shared  the  supervisors’  concerns,  noting  that 
they  believed  that  succeeding  under  NSPS  depended  on  the  quality  of 
their  written  assessments,  rather  than  their  job  performance,  and  that 
their  ratings  could  suffer  if  their  supervisors  did  not  provide  the  pay 
pool  panel  with  well-written  assessments.  In  discussing  the  potential 
influence  that  employees’  and  supervisors’  writing  skills  may  have  on  a 


43The  NSPS  regulations  (5  C.F.R.  §  9901.412(a))  state  that  the  forced  distribution  of  ratings 
(setting  preestablished  limits  for  the  percentage  or  number  of  ratings  that  may  be  assigned 
at  any  level)  is  prohibited. 
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DOD  Has  Not  Yet  Developed  a 
Comprehensive  Action  Plan  to 
Address  Employees’ 
Perceptions  of  the  System 


pay  pool  panel’s  assessment  of  an  employee,  officials  at  seven  of  the 
eight  locations  acknowledged  that  in  some  instances  writing  skills  had 
affected  employees’  ratings  and  could  overshadow  employees’ 
performance. 

•  Discussion  group  participants  at  seven  locations  also  expressed 
frustration  that  employee  ratings  were  potentially  affected  by  the 
extent  to  which  pay  pool  panel  members  have  personal  knowledge  of 
employees  or  understand  the  nature  of  their  work  in  general.  Some 
discussion  group  participants  felt  that  pay  pool  panel  members’ 
personal  knowledge  of  employees  helped  some  employees  receive 
higher  ratings,  while  others  told  us  that  they  felt  that  members  of  the 
pay  pool  panel  were  too  far  removed  from  them  and  lacked  direct 
knowledge  of  the  work  they  performed.  One  employee  believed  that 
individuals  who  were  involved  in  implementing  NSPS  worked  closely 
with  pay  pool  panel  members,  or  were  senior  managers  who  were 
more  likely  to  receive  higher  ratings  under  NSPS  than  others.  Other 
employees  told  us  that  they  were  concerned  about  the  potential  for 
pay  pool  panel  members  to  advocate  in  some  way  for  employees  they 
personally  know — for  example,  by  encouraging  the  pay  pool  panel  to 
contact  a  specific  employee’s  supervisor  to  seek  additional 
information  or  justification  for  a  rating.  As  a  result,  they  felt  that  pay 
pool  panel  members’  personal  knowledge  of  employees  could  benefit 
some  employees,  but  not  others. 

In  our  first  assessment  of  NSPS,  we  recommended  that  DOD  develop  and 
implement  a  specific  action  plan  to  address  employees’  perceptions  of 
NSPS,  based  on  guidance  published  by  OPM  for  conducting  annual 
employee  surveys  and  providing  feedback  to  employees  on  the  results.44 
The  guidance  suggests  that  after  an  agency’s  survey  results  have  been 
reviewed,  the  agency  has  a  responsibility  to  provide  feedback  to 
employees  on  the  results  of  the  survey,  as  well  as  to  let  employees  know 
the  intended  actions  to  address  the  results  and  the  progress  made  on  these 
actions.  Further,  the  guidance  suggests  that  agencies  consider  the 
following  when  developing  action  plans: 

•  the  resources  required; 

•  who  will  be  responsible  for  taking  action; 

•  who  will  be  responsible  for  providing  oversight; 


44Office  of  Personnel  Management,  Annual  Employee  Survey  Guidance  (Washington,  D.C., 
November  2006). 
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•  if  the  individuals  taking  the  action  have  the  necessary  authority  to 
make  things  happen; 

•  what  coordination,  if  any,  is  required,  and  how  it  will  be  accomplished; 
and 

•  how  agencies  will  adjust  given  any  changes  or  delays  in  their  actions. 

Since  then,  in  June  2009,  the  PEO  issued  a  departmentwide  memorandum 
entitled  “Addressing  Key  NSPS  Workforce  Concerns”;  however,  issuance 
of  this  memorandum  does  not  fully  meet  the  intent  of  our  2008 
recommendation.  Specifically,  the  PEO’s  June  2009  memorandum 
summarizes  key  concerns  from  the  department’s  2008  evaluation  of  NSPS, 
summarizes  departmentwide  actions  that  had  been  taken  to  date  to 
address  employees’  concerns  about  the  system,  and  suggests  approaches 
to  enhance  local  efforts  to  address  workforce  concerns.  The  PEO 
identified  five  key  areas  of  concern,  which  are  similar  to  those  identified  in 
our  own  discussion  group  sessions  with  DOD  employees  and  supervisors: 
(1)  performance  communication  and  feedback,  (2)  understanding  of 
performance  management  and  the  pay  pool  process,  (3)  trust  in  the  system 
and  its  processes,  (4)  training  and  information,  and  (5)  the  amount  of  time 
needed  to  fulfill  performance  management  responsibilities.  The  PEO’s 
memorandum  urged  the  components  to  leverage  information  from  the 
department’s  2008  evaluation  of  NSPS  and  focus  on  the  five  areas 
discussed  above  as  they  plan  their  own  actions.  Further,  the  PEO’s 
memorandum  noted  that  DOD  has  taken  some  steps  to  address  employees’ 
concerns  about  NSPS — for  example,  developing  and  fielding  a  pay  pool 
training  course  for  employees  and  rating  officials,  modifying  its 
implementing  issuances  to  require  all  performance  review  authorities  to 
review  pay  pool  panel  results  on  an  annual  basis,  and  providing  guidance 
to  employees  on  the  prohibition  against  the  forced  distribution  of  ratings. 
Issuance  of  the  PEO’s  memorandum  represents  an  important  first  step. 
However,  because  the  memorandum  does  not  specify  actions  the 
department  intends  to  take,  who  will  be  responsible  for  taking  the  action, 
and  timelines  for  addressing  areas  where  employees  express  negative 
perceptions  of  the  system,  it  does  not  fully  meet  the  intent  of  our  2008 
recommendation.  In  developing  an  action  plan,  we  note  that  OPM  recently 
issued  guidance  that  agencies  can  use  in  developing  action  plans  for 
improving  employee  satisfaction.45  According  to  OPM,  action  plans  should 
clearly  (1)  state  the  objectives,  (2)  identify  actions  to  be  taken,  (3)  provide 


45Office  of  Personnel  Management  Memorandum,  “Guidance  on  Employee  Satisfaction 
Action  Planning”  (Aug.  19,  2009). 
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outcome  measures  and  improvement  targets,  and  (4)  describe  how 
progress  will  be  tracked.  In  addition  to  identifying  the  specific  actions  that 
will  be  taken  to  achieve  improvements,  OPM’s  guidance  also  suggests  that 
agencies  specify 

•  time  frames  for  accomplishing  the  actions, 

•  who  will  be  responsible  for  implementing  the  actions, 

•  who  will  be  affected  by  the  actions, 

•  the  resources  required,  and 

•  a  plan  to  communicate  these  actions  to  managers  and  employees. 

We  continue  to  believe  that  developing  and  implementing  a  plan  to 
address  employees’  perceptions  of  NSPS  could  help  DOD  make  changes  to 
the  system  that  could  lead  to  greater  employee  acceptance  and,  ultimately, 
the  system’s  successful  implementation.  Further,  we  note  that  having  such 
a  plan  is  an  approach  that  DOD  could  take  to  involve  employees  in  the 
system’s  implementation — which  is  one  of  the  safeguards  we  previously 
discussed. 


Conclusions 


As  we  noted  in  our  first  assessment,  DOD’s  implementation  of  NSPS 
placed  the  department  at  the  forefront  of  a  significant  transition  facing  the 
federal  government.  However,  toward  the  end  of  this  review,  the  future  of 
NSPS  became  uncertain,  given  the  proposed  legislation  that,  if  enacted, 
would  terminate  the  system  and  require  any  future  system  created  by  DOD 
to  use  safeguards  similar  to  those  discussed  in  our  report,  including 
ensuring  employee  involvement  in  the  system  and  providing  adequate 
training  and  retraining.  In  light  of  the  contingent  nature  surrounding  NSPS 
and  the  possibility  of  implementing  a  different  system,  sustained  and 
committed  leadership  will  be  imperative  to  provide  focused  attention 
necessary  to  implement  any  pay-for-performance  system  within  DOD.  Key 
to  implementing  a  fair,  effective,  and  credible  system  is  including 
safeguards  early  on  in  the  design  of  the  system.  Since  we  issued  our  first 
assessment  of  NSPS  in  2008,  we  note  that  DOD  has  continued  to  take  steps 
to  meet  the  intent  of  each  of  the  safeguards.  However,  with  this  latest 
assessment,  we  note  that  the  department  has  not  implemented  the 
safeguards  systematically;  for  example,  it  has  not  ensured  that  the  training 
provided  to  employees  on  the  system’s  operations  is  effective.  Further, 
DOD  has  not  monitored  how  the  safeguards  specifically  are  implemented 
by  lower-level  organizations  across  the  department.  As  a  result,  decision 
makers  in  DOD  lack  information  that  could  be  used  to  determine  whether 
the  department’s  actions  are  effective  and  whether  the  system  is  being 
implemented  in  a  fair,  equitable,  and  credible  manner.  Additionally,  while 
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DOD  has  gained  experience  operating  under  NSPS,  at  the  time  of  our 
review  it  had  not  yet  developed  an  action  plan  for  addressing  employees’ 
perceptions  of  the  system,  as  we  recommended  in  2008.  As  DOD  moves 
forward  with  implementing  a  pay-for-performance  system — whether  NSPS 
or  another — we  believe  that  it  is  important  for  the  department  to  improve 
upon  its  implementation  of  the  safeguards  and  address  employees’ 
concerns.  Left  unchecked,  these  issues  could  undermine  any  future  human 
capital  reform  efforts  within  DOD. 


Recommendations  for  imPlement  a  fair,  effective,  and  credible  performance  management 

system  for  its  civilian  employees — whether  NSPS  or  another — we 
Executive  Action  recommend  that  the  Secretary  of  Defense  take  the  following  three  actions: 

•  Review  and  evaluate  the  effectiveness  of  the  department’s  training. 

•  Ensure  that  guidance  is  in  place  for  conducting  a  postdecisional 
analysis  that  specifies  what  process  the  components  should  follow  to 
investigate  and  eliminate  potential  barriers  to  fair  and  equitable 
ratings. 

•  Include,  as  part  of  the  department’s  monitoring  of  the  implementation 
of  its  system,  efforts  to  monitor  and  evaluate  how  the  safeguards 
specifically  are  implemented  by  lower-level  organizations  across  the 
department. 


Agency  Comments 
and  Our  Evaluation 


In  September  2009,  we  provided  DOD  with  a  draft  of  this  report  that 
included  three  recommendations  to  better  address  the  safeguards  and 
improve  implementation  of  the  NSPS  performance  management  system. 
Specifically,  we  recommended  that  DOD  (1)  evaluate  NSPS  training, 

(2)  review  and  revise  its  guidance  for  conducting  postdecisional  analysis 
of  NSPS  ratings,  and  (3)  monitor  how  the  safeguards  specifically  are 
implemented.  In  commenting  on  a  draft  of  our  report,  DOD  partially 
concurred  with  our  three  recommendations.  DOD’s  comments  are 
reprinted  in  appendix  III. 

DOD  partially  concurred  with  our  recommendations,  noting  the 
expectation  that  the  Congress  would  require  the  department  to  terminate 
NSPS  by  January  1,  2012,  and  this  action,  in  turn,  would  require  the 
department  to  focus  on  drawing  down  NSPS  in  an  orderly  manner.  DOD 
further  stated  that  it  would  consider  acting  on  our  recommendations  to 
the  extent  they  are  relevant  as  the  department  moves  forward  with  any 
future  performance  management  system.  We  believe  that  this  is  a 
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reasonable  approach.  As  discussed  above,  we  recognize  the  contingent 
nature  surrounding  NSPS  as  a  result  of  provisions  in  the  proposed 
National  Defense  Authorization  Act  for  Fiscal  Year  2010,  which  recently 
passed  both  Houses  of  Congress.  Accordingly,  we  revised  our 
recommendations  to  apply  to  any  future  performance  management  system 
for  the  department’s  civilian  employees — whether  NSPS  or  another 
system.  However,  we  also  note  that  provisions  of  the  proposed  legislation 
would  require  DOD  to  implement  certain  safeguards  and  issue  regulations 
for  that  system  to  provide  a  fair,  credible  and  transparent  performance 
appraisal  system.  We  therefore  continue  to  believe  that  our 
recommendations  have  merit. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees.  We  will  make  copies  available  to  others  upon  request.  The 
report  also  is  available  at  no  charge  on  GAO’s  Web  site  at 
http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact  me 
at  (202)  512-3604  or  farrellb@gao.gov.  Contact  points  for  our  Offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  GAO  staff  who  made  major  contributions  to  the  report  are 
listed  in  appendix  IV. 


Brenda  S.  Farrell 

Director,  Defense  Capabilities  and  Management 


Page  51 


GAO-10-102  Human  Capital 


List  of  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Joseph  I.  Lieberman 
Chairman 

The  Honorable  Susan  M.  Collins 
Ranking  Member 

Committee  on  Homeland  Security  and  Governmental  Affairs 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  Howard  P.  McKeon 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Edolphus  Towns 
Chairman 

The  Honorable  Darrell  Issa 
Ranking  Member 

Committee  on  Oversight  and  Government  Reform 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


As  with  our  first  assessment  of  the  National  Security  Personnel  System 
(NSPS)  in  2008, 1  we  limited  our  scope  in  conducting  this  review  to  the 
performance  management  aspect  of  NSPS.  Therefore,  we  addressed 
neither  performance  management  of  the  Senior  Executive  Service  at  the 
Department  of  Defense  (DOD)  nor  other  aspects  of  NSPS,  such  as 
classification  and  pay. 


Determining 
Implementation  of 
Safeguards  and  Monitoring 
Their  Implementation 


To  determine  the  extent  to  which  DOD  has  implemented  safeguards  as 
part  of  the  NSPS  performance  management  system  and  monitored  the 
implementation  of  the  safeguards,  we  used  the  following  safeguards, 
which  we  also  reported  on  in  our  2008  review: 


•  Involve  employees,  their  representatives,  and  other  stakeholders  in  the 
design  of  the  system,  to  include  employees  directly  involved  in 
validating  any  related  implementation  of  the  system. 


•  Assure  that  the  agency’s  performance  management  systems  link 
employee  objectives  to  the  agency’s  strategic  plan,  related  goals,  and 
desired  outcomes. 


•  Provide  adequate  training  and  retraining  for  supervisors,  managers, 
and  employees  in  the  implementation  and  operation  of  the 
performance  management  system. 

•  Provide  a  process  for  ensuring  ongoing  performance  feedback  and 
dialogue  between  supervisors,  managers,  and  employees  throughout 
the  appraisal  period,  and  for  setting  timetables  for  review. 

•  Implement  a  pay-for-performance  evaluation  system  to  better  link 
individual  pay  to  performance,  and  provide  an  equitable  method  for 
appraising  and  compensating  employees. 

•  Assure  that  certain  predecisional  internal  safeguards  exist  to  help 
achieve  consistency,  equity,  nondiscrimination,  and  nonpoliticization 
of  the  performance  management  process  (e.g.,  independent 
reasonableness  reviews  by  a  third  party  or  reviews  of  performance 
rating  decisions,  pay  determinations,  and  promotions  before  they  are 


^AO,  Human  Capital:  DOD  Needs  to  Improve  Implementation  of  and  Address  Employee 
Concerns  about  Its  National  Security  Personnel  System,  GAO-08-773  (Washington,  D.C.: 
Sept.  10,  2008). 
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finalized  to  ensure  that  they  are  merit-based,  as  well  as  pay  panels  who 
consider  the  results  of  the  performance  appraisal  process  and  other 
information  in  connection  with  final  pay  decisions). 

•  Assure  that  there  are  reasonable  transparency  and  appropriate 
accountability  mechanisms  in  connection  with  the  results  of  the 
performance  management  process,  to  include  reporting  periodically  on 
internal  assessments  and  employee  survey  results  relating  to 
performance  management  and  individual  pay  decisions  while 
protecting  individual  confidentiality. 

•  Assure  that  performance  management  results  in  meaningful 
distinctions  in  individual  employee  performance. 

•  Provide  a  means  for  ensuring  that  adequate  agency  resources  are 
allocated  for  the  design,  implementation,  and  administration  of  the 
performance  management  system. 

To  assess  implementation  of  the  safeguards,  we  reviewed  the  legislative 
requirements  and  obtained  and  analyzed  regulations  and  other  guidance 
for  implementing  the  NSPS  performance  management  system.  We  also 
obtained  and  analyzed  other  documents,  such  as  DOD’s  rating  results  and 
reconsideration  statistics,  for  the  2007  and  2008  NSPS  performance 
management  cycles.2  We  also  interviewed  knowledgeable  officials  in 
DOD’s  NSPS  Program  Executive  Office  and  the  NSPS  program  offices  of 
the  four  components — the  Army,  the  Air  Force,  the  Navy,  and  the  Fourth 
Estate3 — to  obtain  a  comprehensive  understanding  of  their  efforts  to 
implement  NSPS  and  each  of  the  safeguards,  as  well  as  the  processes, 
procedures,  and  controls  used  for  monitoring  and  overseeing 
implementation  of  the  system.  In  addition,  we  conducted  site  visits  to 
select  organizations  located  outside  the  continental  United  States  to 
assess  implementation  of  the  safeguards.  To  allow  for  appropriate 
representation  by  each  component,  we  visited  two  organizations  per 
component,  or  eight  organizations  in  total.  The  organizations  we  visited 


2GAO  has  not  independently  verified  the  reliability  of  DOD’s  reported  reconsideration 
statistics. 

3The  Department  of  the  Navy’s  NSPS  policies  encompass  Marine  Corps  civilians.  The 
Fourth  Estate  includes  all  organizational  entities  in  DOD  that  are  not  in  the  military 
departments  or  the  combatant  commands.  Examples  of  Fourth  Estate  entities  are  the 
Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Office  of  the  DOD  Inspector  General, 
the  defense  agencies,  and  DOD  field  activities. 
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were  selected  based  on  a  number  of  factors,  such  as  the  presence  of  a 
large  number  or  concentrated  group  of  civilian  employees  under  NSPS 
and,  when  possible,  the  presence  of  employees  who  had  converted  to 
NSPS  under  both  spirals  1  and  2.  We  focused  our  efforts  for  this  review  on 
visiting  organizations  located  outside  the  continental  United  States 
because  our  2008  review  focused  on  assessing  implementation  of  NSPS 
and  the  safeguards  at  locations  that  were  geographically  distributed 
throughout  the  United  States.  We  elected  to  focus  our  site  visits  in 
Germany  and  Hawaii  because  of  the  civilian  employees  located  outside  the 
continental  United  States  who  had  converted  to  NSPS  at  the  time  we 
initiated  our  review,  more  than  half  were  located  in  either  Germany  or 
Hawaii.4  Also,  we  wanted  to  determine  whether  civilian  employees  located 
outside  the  continental  United  States  were  experiencing  any  unique 
problems  or  challenges  with  the  system.  In  Germany,  the  organizations  we 
visited  were  the  5th  Signal  Command;  the  435th  Air  Base  Wing;  the 
Defense  Finance  and  Accounting  Service;  and  the  George  C.  Marshall 
European  Center  for  Security  Studies,  part  of  the  Defense  Security 
Cooperation  Agency.  In  Hawaii,  the  organizations  we  visited  were  the 
Commander,  Navy  Region  Hawaii;  Headquarters,  Pacific  Air  Force;  the 
Naval  Facilities  Engineering  Command,  Hawaii;  and  the  U.S.  Army  Corps 
of  Engineers,  Honolulu  District.  For  each  of  the  organizations  we  visited, 
we  met  with  or  interviewed  the  performance  review  authority,  pay  pool 
managers,  pay  pool  panel  members,  rating  officials,  and  the  NSPS  program 
manager  or  transition  team,  among  others,  to  discuss  the  steps  they  have 
taken  to  implement  the  safeguards  or  otherwise  ensure  the  fairness, 
effectiveness,  and  credibility  of  NSPS.  To  assess  the  organizations’ 
implementation  of  the  safeguards,  we  compared  and  contrasted  the 
information  obtained  during  our  interviews  and  supplemented  this 
testimonial  evidence  with  the  other  relevant  documentation  we  obtained, 
such  as  the  organizations’  pay  pool  business  rules,5  lessons  learned,  and 
training  materials. 


According  to  the  Program  Executive  Office,  as  of  September  2008,  about  85  percent  of  the 
civilian  employees  under  NSPS  were  located  in  the  continental  United  States,  whereas 
about  15  percent  were  located  outside  the  continental  United  States.  Of  the  civilian 
employees  under  NSPS  located  outside  the  continental  United  States,  about  54  percent 
were  in  either  Germany  or  Hawaii. 

^Business  rules  are  the  policies  that  govern  a  pay  pool’s  operations.  They  may  specify,  for 
example,  the  pay  pool  panel’s  structure,  roles  and  responsibilities,  standards  of  conduct, 
and  the  processes  used  for  reconciling  employee  ratings  and  allocating  shares  under  NSPS. 
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Determining  DOD  Civilian 
Employees’  Perceptions  of 
NSPS 


Analysis  of  DOD  Survey 
Results 


To  determine  how  DOD  civilian  employees  perceive  NSPS,  we  analyzed 
two  sources  of  employee  perceptions  or  attitudes.  First,  we  analyzed  the 
results  of  DOD’s  survey  of  civilian  employees  to  identify  employee 
perceptions  of  NSPS  and  examine  whether  and  how  these  perceptions 
may  be  changing  over  time.  Second,  we  conducted  small  group 
discussions  with  civilian  employees  who  had  converted  to  NSPS  and 
administered  a  short  questionnaire  to  the  participants  at  each  of  the  eight 
organizations  we  visited.  As  with  our  first  assessment  of  NSPS,  our  overall 
objective  in  using  the  discussion  group  approach  was  to  obtain  employees’ 
perceptions  about  NSPS  and  its  implementation  thus  far. 

We  analyzed  the  results  of  the  Defense  Manpower  Data  Center’s  (DMDC) 
Status  of  Forces  Survey6  of  Civilian  Employees — including  the  May  2006, 
November  2006,  May  2007,  and  February  2008  administrations — to  gauge 
employee  attitudes  toward  NSPS  and  performance  management  in  general 
and  identify  indications  of  movement  or  trends  in  employee  perceptions.7 
As  we  reported  in  September  2008,  we  have  reviewed  the  results  of  prior 
administrations  of  DMDC  surveys  and  found  the  survey  results,  including 
the  results  of  the  Status  of  Forces  Survey  of  Civilian  Employees, 
sufficiently  reliable  to  use  for  several  GAO  engagements.  However,  to 
understand  the  nature  of  any  changes  that  were  made  to  its  survey 
methods  for  administering  the  survey  for  2008  as  compared  with  previous 
administrations,  we  also  received  responses  to  written  questions  from  and 
discussed  these  data  with  officials  at  DMDC.  Based  on  these  responses 
and  discussions,  we  determined  that  DMDC’s  survey  data  remain 


6The  Status  of  Forces  Survey  is  a  series  of  Web-based  surveys  of  the  total  force  that  allows 
DOD  to  (1)  evaluate  existing  programs/policies,  (2)  establish  baselines  before 
implementing  new  programs/policies,  and  (3)  monitor  progress  of  programs/policies  and 
their  effects  on  the  total  force. 

DMDC  has  conducted  large-scale,  departmentwide  surveys  of  active  military  personnel 
since  2002,  called  the  Status  of  Forces  Active  Duty  Survey.  DMDC  has  also  conducted 
surveys  of  reserve  military  personnel  for  DOD  called  the  Status  of  Forces  Reserve  Survey. 
Since  2003,  DMDC  has  administered  its  Status  of  Forces  Survey  of  Civilian  Employees, 
which  includes  questions  about  compensation,  performance,  and  personnel  processes.  All 
surveys  include  outcome  or  “leading  indicator”  measures  such  as  overall  satisfaction, 
retention  intention,  and  perceived  readiness,  as  well  as  demographic  items  needed  to 
classify  individuals  into  various  subpopulations.  Regular  administrations  of  the  Status  of 
Forces  Survey  of  Civilian  Employees  occurred  every  6  months  from  October  2004  through 
November  2006,  while  annual  administrations  commenced  in  2007. 
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GAO’s  Discussion  Groups  with 
DOD  Civilian  Employees  under 
NSPS 


sufficiently  reliable  for  the  purpose  of  our  reports  on  DOD  civilian 
employees’  perceptions  of  NSPS.8 

We  also  conducted  small  group  discussions  with  DOD  civilian  employees 
and  administered  a  short  questionnaire  during  site  visits  in  February  and 
March  2009.  Specifically,  we  conducted  two  discussion  groups — one  with 
nonsupervisory  employees  and  another  with  supervisory  employees — at 
each  of  the  eight  locations  we  visited,  for  a  total  of  16  discussion  groups. 
As  with  our  first  assessment  of  NSPS  in  2008,  our  objective  in  using  this 
approach  was  to  obtain  employees’  perceptions  about  NSPS  and  its 
implementation  thus  far  because  discussion  groups  are  intended  to 
provide  in-depth  information  about  participants’  reasons  for  holding 
certain  attitudes  about  specific  topics  and  to  offer  insights  into  the  range 
of  concerns  about  and  support  for  an  issue.  Further,  in  conducting  our 
discussion  groups,  our  intent  was  to  achieve  saturation — the  point  at 
which  we  were  no  longer  hearing  new  information. 

As  we  previously  reported,  our  discussion  groups  were  not  designed  to 
(1)  demonstrate  the  extent  of  a  problem  or  to  generalize  the  results  to  a 
larger  population,  (2)  develop  a  consensus  to  arrive  at  an  agreed-upon 
plan  or  make  decisions  about  what  actions  to  take,  or  (3)  provide 
statistically  representative  samples  or  reliable  quantitative  estimates. 
Instead,  our  discussion  groups  provide  in-depth  information  about 
participants’  reasons  for  holding  certain  attitudes  about  specific  topics  and 
offer  insights  into  the  range  of  concerns  about  and  support  for  an  issue. 
Although  the  results  of  our  discussion  sessions  are  not  generalizable  to  the 
entire  NSPS  civilian  population,  the  composition  of  our  discussion  groups 
was  designed  to  ensure  that  we  spoke  with  employees  from  each  of  the 
four  components  at  locations  outside  the  continental  United  States. 
Because  supervisory  and  nonsupervisory  employees  have  distinct  roles 
with  respect  to  NSPS,  we  held  separate  discussion  sessions  for  these 
groups. 

To  select  the  discussion  group  participants,  we  requested  that  the 
organizations  we  visited  provide  us  with  lists  of  employees  who  had 
converted  to  NSPS.  From  the  lists  provided,  we  selected  participants 
based  on  their  supervisory  and  nonsupervisory  status.  To  ensure 


sIn  our  September  2008  report,  we  identified  areas  for  improvement  with  regard  to  DOD’s 
survey  results — for  example,  use  of  nonresponse  analysis,  which  is  a  good  survey  research 
practice,  to  clarify  whether  those  employees  who  did  not  respond  to  DOD’s  surveys  may 
provide  substantively  different  answers  than  those  who  did  respond. 
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maximum  participation  of  the  selected  employees,  we  randomly  selected 
up  to  20  participants  from  each  group  with  the  goal  of  meeting  with  8  to  12 
individuals  in  each  discussion  group  and  provided  the  employee  names 
and  a  standard  invitation  to  GAO’s  points  of  contact  to  disseminate  to  the 
employees.  At  the  majority  of  locations,  we  reached  our  goal  of  meeting 
with  8  to  12  individuals  in  each  discussion  group;  however,  since 
participation  was  not  compulsory,  in  two  instances  we  did  not  reach  our 
goal  of  8  participants  per  discussion  group.  Table  6  provides  information 
on  the  composition  of  our  discussion  groups. 


Table  6:  Composition  of  Discussion  Groups 

Discussion  group  participants 

Total  NSPS 
employees 
assigned  to  the 
pay  pool  visited8 

Organization 

Nonsupervisory 

employees 

Supervisory 

employees 

Total  discussion 
group  participants 

Army 

5th  Signal  Command,  Funari  Barracks,  Germany 

11 

10 

21 

557 

U.S .  Army  Corps  of  Engineers,  H onolulu  District, 
FortShafter,  Hawaii 

7 

9 

16 

246 

Air  Force 

435th  Air  Base  Wing,  Ramstein  Air  Base, 

Germany 

13 

9 

22 

314 

Headquarters,  Pacific  Air  Force,  Hickam  Air  Force 
Base,  Hawaii 

9 

9 

18 

100 

Navy 

Commander,  Navy  Region  Hawaii,  Pearl  Harbor, 
Hawaii 

12 

15 

27 

320 

Naval  Facilities  Engineering  Command,  Hawaii, 
Pearl  Harbor,  Hawaii 

11 

10 

21 

97 

Fourth  Estate 

Defense  Finance  and  Accounting  Service, 
Kaiserslautern,  Germany 

12 

4 

16 

1,497 

George  C.  Marshall  European  Center  for  Security 
Studies,  Garmisch-Partenkirchen,  Germany 

10 

13 

23 

434 

Total 

85 

79 

164 

3,565 

Source:  GAO  (analysis)  and  DOD  (number  of  employees  assigned  to  the  respective  pay  pool). 


The  totals  listed  include  the  number  of  civilian  employees  each  organization  rated  during  the  2008 
NSPS  performance  management  cycle. 


To  facilitate  our  discussion  groups,  we  developed  a  discussion  guide  to 
help  the  moderator  in  addressing  several  topics  related  to  employees’ 
perceptions  of  the  NSPS  performance  management  system.  These  topics 
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include  employees’  overall  perception  of  NSPS  and  the  rating  process,  the 
training  they  received  on  NSPS,  the  communication  they  have  had  with 
their  supervisors,  positive  aspects  they  perceive  of  NSPS,  and  any  changes 
they  would  make  to  the  system,  among  others.  Each  discussion  group  was 
scheduled  for  a  2-hour  period  and  began  with  the  GAO  moderator  greeting 
the  participants,  describing  the  purpose  of  the  study,  and  explaining  the 
procedures  for  the  discussion  group.  Participants  were  assured  that  all  of 
their  comments  would  be  discussed  in  the  aggregate  or  as  part  of  larger 
themes  that  emerged.  The  GAO  moderator  asked  participants  open-ended 
questions  related  to  NSPS,  while  at  least  one  other  GAO  analyst  observed 
the  discussion  group  and  took  notes.  Following  the  conclusion  of  all  our 
discussion  group  sessions,  we  performed  content  analysis  of  the  sessions 
in  order  to  identify  the  themes  that  emerged  and  to  summarize  the 
participants’  perceptions  of  NSPS.  We  reviewed  responses  from  several  of 
the  discussion  groups  and  created  a  list  of  themes  and  subtheme 
categories.  We  then  reviewed  the  comments  from  each  of  the  16 
discussion  groups  and  assigned  each  comment  to  the  appropriate 
category,  which  was  agreed  upon  by  two  analysts.  If  agreement  was  not 
reached  on  a  comment’s  placement  in  a  category,  another  analyst 
reconciled  the  issue  by  placing  the  comment  in  either  one  or  more  of  the 
categories.  The  responses  in  each  category  were  then  used  in  our 
evaluation  and  discussion  of  how  civilian  employees  perceive  NSPS. 

Following  each  discussion  group  we  administered  a  questionnaire  to  the 
participants  to  obtain  further  information  on  their  background,  tenure 
with  the  federal  government  and  DOD,  and  attitudes  toward  NSPS.  We 
received  questionnaires  from  164  discussion  group  participants.  In 
addition  to  collecting  demographic  data  from  participants  for  the  purpose 
of  reporting  with  whom  we  spoke  (see  table  7),  the  purpose  of  our 
questionnaire  was  to  (1)  collect  information  from  participants  that  could 
not  easily  be  obtained  through  discussion,  for  example,  information 
participants  may  have  been  uncomfortable  sharing  in  a  group  setting,  and 
(2)  collect  some  of  the  same  data  found  in  past  DOD  surveys.  Specifically, 
the  questionnaire  included  questions  designed  to  obtain  employees’ 
perceptions  of  NSPS  as  compared  to  their  previous  personnel  system,  the 
accuracy  with  which  they  felt  their  ratings  reflected  their  performance, 
and  management’s  methods  for  conveying  overall  rating  information. 

Since  the  questionnaire  was  used  to  collect  supplemental  information  and 
was  administered  solely  to  the  participants  of  our  discussion  groups,  the 
results  represent  the  opinions  of  only  those  employees  and  cannot  be 
projected  across  DOD,  a  component,  or  any  single  pay  pool  we  visited. 
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Table  7:  Composition  of  Discussion  Groups  by  Demographic  Category  per  Component 


Component 

Category 

Army 

Air  Force 

Navy 

Fourth  Estate 

Total 

Male 

22 

25 

21 

17 

85 

Female 

15 

15 

27 

22 

79 

Total 

37 

40 

48 

39 

164 

American  Indian  or  Alaskan  Native 

0 

1 

0 

0 

1 

Asian 

14 

4 

21 

1 

40 

Black/African  American 

4 

2 

1 

4 

11 

Hispanic 

0 

1 

2 

3 

6 

Native  Hawaiian  or  other  Pacific  Islander 

0 

0 

7 

0 

7 

White 

16 

28 

14 

31 

89 

Other 

3 

4 

3 

0 

10 

Total 

37 

40 

48 

39 

164 

Source:  GAO. 

Note:  Participants  voluntarily  self-reported  demographic  information  in  our  questionnaire. 


We  visited  or  contacted  the  following  organizations  during  our  review: 


Department  of  Defense 


Defense  Finance  and  Accounting  Service,  Indianapolis,  Indiana;  Rome, 
New  York;  Columbus,  Ohio;  and  Kaiserslautern,  Germany 
Defense  Manpower  Data  Center,  Arlington,  Virginia 
Defense  Security  Cooperation  Agency,  Arlington,  Virginia 

•  George  C.  Marshall  European  Center  for  Security  Studies, 
Garmisch-Partenkirchen,  Germany 
Equal  Employment  Opportunity  Office,  Arlington,  Virginia 
Fourth  Estate  NSPS  Program  Management  Office,  Arlington,  Virginia 
NSPS  Program  Executive  Office,  Arlington,  Virginia 


Department  of  the  Army 


5th  Signal  Command,  Funari  Barracks,  Germany 
Civilian  Personnel  Evaluation  Agency,  Alexandria,  Virginia 
NSPS  Program  Management  Office,  Alexandria,  Virginia 
U.S.  Army  Corps  of  Engineers 

•  Headquarters,  Washington,  D.C. 

•  Honolulu  District,  Fort  Shatter,  Hawaii 

•  Pacific  Ocean  Division,  Fort  Shafter,  Hawaii 
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Department  Of  the  Air  •  435th  Air  Base  Wing,  Ramstein  Air  Base,  Germany 

Force  *  Headquarters,  Pacific  Air  Force,  Hickam  Air  Force  Base,  Hawaii 

•  NSPS  Program  Office,  Arlington,  Virginia 


Department  of  the  Navy 


•  Commander,  Navy  Region  Hawaii,  Pearl  Harbor,  Hawaii 

•  Naval  Facilities  Engineering  Command,  Hawaii,  Pearl  Harbor,  Hawaii 

•  NSPS  Program  Office,  Navy  Yard,  Washington,  D.C. 

•  Office  of  Civilian  Human  Resources,  Navy  Yard,  Washington,  D.C. 

We  conducted  this  performance  audit  from  November  2008  through 
September  2009  in  accordance  with  generally  accepted  government 
auditing  standards.  Those  standards  require  that  we  plan  and  perform  the 
audit  to  obtain  sufficient  and  appropriate  evidence  to  provide  a  reasonable 
basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We 
believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives. 
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Department  of  Defense  (DOD)  civilian  employees  who  receive 
performance  ratings  under  the  National  Security  Personnel  System  (NSPS) 
have  the  option  of  challenging  their  ratings  through  a  formal  process 
known  as  reconsideration.  The  reconsideration  process  is  the  sole  and 
exclusive  agency  administrative  process  for  nonbargaining  unit 
employees  to  challenge  their  ratings.1  However,  DOD’s  NSPS  regulations 
also  allow  for  alternative  dispute  resolution  techniques,  such  as  mediation 
or  interest-based  problem  solving,  to  be  pursued  at  any  time  during  the 
reconsideration  process  consistent  with  component  policies  and 
procedures.  Under  the  reconsideration  process,  employees  may  challenge 
their  ratings  of  record  or  individual  job  objective  ratings;  employees 
cannot  challenge  their  performance  payout,  the  number  of  shares 
assigned,  the  share  value,  or  the  distribution  of  their  performance  payout 
between  salary  increase  and  bonus,  nor  can  they  challenge  their 
recommended  ratings  of  record,  interim  reviews,  or  applicable  closeout 
assessments.  In  addition,  employees  who  allege  that  their  performance 
ratings  are  based  on  prohibited  discrimination  or  reprisal  may  not  use  the 
reconsideration  process;  rather,  such  allegations  are  to  be  processed 
through  the  department’s  equal  employment  opportunity  discrimination 
complaint  procedure. 

Employees  who  wish  to  challenge  their  rating  have  10  calendar  days  from 
the  receipt  of  their  ratings  of  record  to  submit  written  requests  for 
reconsideration  to  their  pay  pool  managers.  Within  15  calendar  days  of  the 
pay  pool  manager’s  receipt  of  an  employee’s  request  for  reconsideration, 
the  pay  pool  manager  must  render  a  written  decision  that  includes  a  brief 
explanation  of  the  basis  of  the  decision.  The  pay  pool  manager’s  decision 
is  final,  unless  the  employee  seeks  further  reconsideration  from  the 
performance  review  authority.  Specifically,  if  the  employee  is  dissatisfied 
with  the  pay  pool  manager’s  decision,  or  if  none  is  provided  within  the 
prescribed  time  frames,  the  employee  may  submit  a  written  request  for 
final  review  by  the  performance  review  authority  or  his  or  her  designee. 
This  request  must  be  submitted  within  5  calendar  days  of  receipt  of  the 
pay  pool  manager’s  decision  or  within  5  calendar  days  of  the  date  the 


hn  contrast,  negotiated  grievance  procedures  are  the  exclusive  administrative  procedures 
for  bargaining  unit  employees.  However,  if  a  negotiated  grievance  procedure  is  not 
available  to  a  bargaining  unit  employee  or  challenging  a  rating  of  record  or  job  objective 
rating  is  outside  the  scope  of  the  employee’s  negotiated  grievance  procedure,  a  bargaining 
unit  employee  may  challenge  his  or  her  rating  of  record  or  job  objective  rating  in 
accordance  with  the  DOD’s  NSPS  regulations  and  implementing  issuances.  As  of  May  2009, 
DOD  reported  that  there  were  approximately  685  bargaining  unit  employees  and  18 
certified  bargaining  units  representing  NSPS  employees. 
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decision  should  have  been  rendered.  The  performance  review  authority 
then  is  allotted  15  calendar  days  from  receipt  of  the  written  request  from 
the  employee  to  make  a  decision,  which  is  final.  If  the  final  decision  is  to 
change  the  rating  of  record  or  job  objective  rating,  the  revised  rating  takes 
the  place  of  the  original  one,  and  a  revised  performance  appraisal  is 
prepared  for  the  employee. 

According  to  DOD’s  2008  evaluation  report,  for  the  2007  NSPS 
performance  management  cycle,2  2,302  civilian  employees  out  of  the 
100,465  employees  who  were  rated  under  NSPS  elected  to  file  a  request  for 
reconsideration  of  their  ratings,  and  of  these,  about  33  percent  of  the 
requests  were  granted.  For  the  2008  NSPS  performance  management 
cycle,3  according  to  the  NSPS  Program  Executive  Office,  as  of  June  2009, 
4,296  civilian  employees  out  of  the  170,149  employees  who  were  rated 
under  NSPS  elected  to  file  requests  for  reconsideration  of  their  ratings, 
and  of  these,  about  52  percent  of  the  requests  were  granted.4 


2DOD  civilian  employees  under  NSPS  received  their  payout  for  the  2007  NSPS  performance 
management  cycle  in  January  2008. 

3DOD  civilian  employees  under  NSPS  received  their  payout  for  the  2008  NSPS  performance 
management  cycle  in  January  2009. 

4GAO  has  not  independently  verified  the  reliability  of  DOD’s  reported  reconsideration 
statistics. 
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DEPARTMENT  OF  DEFENSE 
NATIONAL  SECURITY  PERSONNEL  SYSTEM 
PROGRAM  EXECUTIVE  OFFICE 
1400  KEY  BOULEVARO  SUITE  B200 
ARLINGTON.  VA  22200-5144 

October  23,  2009 

Ms.  Brenda  S.  Farrell 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Ms.  Farrell: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  dratf  report,  "HUMAN 
CAPITAL:  Continued  Monitoring  of  Safeguards  Is  Needed  and  Opportunities  Exist  to  Address 
Employee  Concerns  about  the  National  Security  Personnel  System"  dated  September  2 1 . 2009 
(GAO  Code  35I277/GAO-10-I02).  We  thank  you  for  the  opportunity  to  review  and  comment. 

Our  own  internal  program  evaluation  work  and  other  feedback  have  pointed  us  to  many 
of  the  same  issues  that  your  report  cites.  But  for  our  expectation  that  Congress  will  require  the 
Department  to  terminate  the  National  Security  Personnel  System  (NSPS)  by  January  1,  2012,  we 
would  concur  in  your  basic  recommendations.  They  align  with  actions  we  have  said  we  would 
take  as  NSPS  becomes  a  more  fully  mature  human  resources  system. 

As  you  report,  the  Department  has  listened  to  the  workforce  and  made  adjustments  since 
first  implementing  Spiral  1.1.  For  example,  we  modified  system  rules  to  increase  accountability 
at  senior  levels,  added  NSPS  training  and  reference  material  that  individuals  could  access  for 
themselves  on  line,  and  improved  appraisal  tools.  Now  that  most  organizations  have  had  at  least 
two  years  of  experience  with  NSPS,  this  year  would  have  been  the  time  to  see  if  the  fundamental 
safeguards  for  fair,  objective,  rigorous  ratings  (c.g.,  the  multiple  levels  of  review  and  standard 
rating  criteria)  were  sound  or  created  their  own,  undesirable  complications  for  us  to  correct. 

Congress’  mandate  will  require  that  we  focus  on  drawing  down  NSPS  in  an  orderly  way. 
Wc  therefore  will  consider  acting  on  your  three  recommendations  to  the  extent  they  are  relevant 
as  the  Department  moves  forward  with  any  new  systems.  We  appreciate  your  staffs  efforts  to 
review  and  report  on  NSPS  safeguards,  with  sensitivity  to  the  cultural  change  involved  and  the 
time  and  evolutionary  actions  needed  to  complete  such  a  transformation.  As  we  noted  last  year, 
it  is  an  enormous  challenge  to  ensure  a  performance  management  system  is  rigorous  and 
consequential  as  well  as  fair,  transparent,  and  well  accepted. 


Sincerely,  yj 


Brad  Bunn 

Program  Executive  Officer 

Enclosure 


<Pi\15F>5 


Page  64 


GAO-10-102  Human  Capital 


Appendix  III:  Comments  from  the  Department 
of  Defense 


ENCLOSURE 

GAO  DRAFT  REPORT  DATED  SEPTEMBER  21,  2009 
GAO  CODE  351277/GAO- 10- 102 

"HUMAN  CAPI I  AL:  Continued  Monitoring  of  Safeguards  Is  Needed  and  Opportunities 
Exist  to  Address  Employee  Concerns  about  the  National  Security  Personnel  System" 

DEPARTMENT  OF  DEFENSE  RESPONSES  TO  RECOMMENDATIONS 

RECOMMENDATION  I:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
National  Security  Personnel  System  (NSPS)  Senior  Executive  to  review  and  evaluate  the 
Department’s  NSPS  training  to  enhance  the  effectiveness  of  the  training. 

POD  RESPONSE:  Partially  concur. 

If  NSPS  were  not  expected  to  be  terminated,  we  agree  that  it  would  be  timely  to  assess  NSPS 
training  in  terms  of  outcomes  as  well  as  ongoing  needs.  The  DOD  training  evaluation  strategy 
included  this  mid-term  step. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
NSPS  Senior  Executive  to  review  and  revise  the  Department’s  guidance  for  conducting  a 
postdecisional  analysis  to  specify  what  process  the  components  should  follow  to  investigate  and 
eliminate  potential  barriers  to  fair  and  equitable  ratings. 

DOD  RESPONSE:  Partially  concur. 

If  NSPS,  a  pay  for  performance  system,  were  not  expected  to  be  terminated,  the  Department 
would  add  process  suggestions  to  help  Components  adapt  their  experience  with  annual  barrier 
analysis  methods  for  the  Equal  Employment  Opportunity  program,  to  rating  and  payout  results. 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
NSPS  Senior  Executive  to  include,  as  a  part  of  the  Department’s  monitoring  of  the 
implementation  of  the  NSPS  performance  management  system,  efforts  to  monitor  and  evaluate 
how  the  safeguards  specifically  are  implemented  by  lower-level  organizations  across  the 
Department. 

DOD  RESPONSE:  Partially  concur. 

If  NSPS  were  not  expected  to  be  terminated,  the  Department  would  expand  on  the  attention 
given  to  how  organizations  implement  and  carry  out  NSPS  performance  management  safeguards, 
in  the  DoD  human  capital  accountability  program.  The  nature  of  foture  performance 
management  systems  will  guide  how  the  Department  monitors  them. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 
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U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 
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U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
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